2023-02-08 23:53:55,452 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-08 23:53:55,684 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d 2023-02-08 23:53:55,698 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-08 23:53:55,699 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197, deleteOnExit=true 2023-02-08 23:53:55,699 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-08 23:53:55,699 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/test.cache.data in system properties and HBase conf 2023-02-08 23:53:55,700 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/hadoop.tmp.dir in system properties and HBase conf 2023-02-08 23:53:55,700 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/hadoop.log.dir in system properties and HBase conf 2023-02-08 23:53:55,701 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-08 23:53:55,701 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-08 23:53:55,701 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-08 23:53:55,821 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-08 23:53:56,204 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-08 23:53:56,208 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-08 23:53:56,208 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-08 23:53:56,209 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-08 23:53:56,209 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-08 23:53:56,209 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-08 23:53:56,209 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-08 23:53:56,210 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-08 23:53:56,210 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-08 23:53:56,210 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-08 23:53:56,210 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/nfs.dump.dir in system properties and HBase conf 2023-02-08 23:53:56,211 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/java.io.tmpdir in system properties and HBase conf 2023-02-08 23:53:56,211 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-08 23:53:56,211 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-08 23:53:56,211 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-08 23:53:56,689 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-08 23:53:56,693 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-08 23:53:56,965 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-08 23:53:57,131 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-08 23:53:57,146 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 23:53:57,182 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 23:53:57,213 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/java.io.tmpdir/Jetty_localhost_44491_hdfs____h51z7a/webapp 2023-02-08 23:53:57,365 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44491 2023-02-08 23:53:57,376 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-08 23:53:57,376 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-08 23:53:57,762 WARN [Listener at localhost/38079] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 23:53:57,868 WARN [Listener at localhost/38079] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-08 23:53:57,886 WARN [Listener at localhost/38079] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 23:53:57,892 INFO [Listener at localhost/38079] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 23:53:57,897 INFO [Listener at localhost/38079] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/java.io.tmpdir/Jetty_localhost_45231_datanode____.95bs16/webapp 2023-02-08 23:53:57,993 INFO [Listener at localhost/38079] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45231 2023-02-08 23:53:58,363 WARN [Listener at localhost/33685] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 23:53:58,376 WARN [Listener at localhost/33685] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-08 23:53:58,380 WARN [Listener at localhost/33685] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 23:53:58,382 INFO [Listener at localhost/33685] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 23:53:58,393 INFO [Listener at localhost/33685] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/java.io.tmpdir/Jetty_localhost_42467_datanode____bukf1a/webapp 2023-02-08 23:53:58,495 INFO [Listener at localhost/33685] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42467 2023-02-08 23:53:58,504 WARN [Listener at localhost/39373] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 23:53:58,522 WARN [Listener at localhost/39373] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-08 23:53:58,524 WARN [Listener at localhost/39373] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 23:53:58,526 INFO [Listener at localhost/39373] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 23:53:58,530 INFO [Listener at localhost/39373] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/java.io.tmpdir/Jetty_localhost_44777_datanode____.9mscle/webapp 2023-02-08 23:53:58,632 INFO [Listener at localhost/39373] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44777 2023-02-08 23:53:58,642 WARN [Listener at localhost/37973] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 23:53:58,844 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6cb358a208cfdeba: Processing first storage report for DS-78636c1c-1e4c-456a-9f65-6d5b741dfa42 from datanode c96d9a8d-7822-404e-9335-ec30f9bdbf87 2023-02-08 23:53:58,846 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6cb358a208cfdeba: from storage DS-78636c1c-1e4c-456a-9f65-6d5b741dfa42 node DatanodeRegistration(127.0.0.1:45117, datanodeUuid=c96d9a8d-7822-404e-9335-ec30f9bdbf87, infoPort=34387, infoSecurePort=0, ipcPort=37973, storageInfo=lv=-57;cid=testClusterID;nsid=1459456370;c=1675900436765), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-08 23:53:58,846 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x12f85878f61cda94: Processing first storage report for DS-a61b8f87-3dea-401a-b95a-91f3efd93fae from datanode d39c9230-801b-44ca-bfae-38ddf4db7d5a 2023-02-08 23:53:58,846 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x12f85878f61cda94: from storage DS-a61b8f87-3dea-401a-b95a-91f3efd93fae node DatanodeRegistration(127.0.0.1:40815, datanodeUuid=d39c9230-801b-44ca-bfae-38ddf4db7d5a, infoPort=44023, infoSecurePort=0, ipcPort=39373, storageInfo=lv=-57;cid=testClusterID;nsid=1459456370;c=1675900436765), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 23:53:58,846 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x23b09932566600c4: Processing first storage report for DS-0dbaaba5-6e2a-4661-a4e7-d380caea9a6f from datanode e53af517-ccea-4786-a9f1-d1d57e9a860e 2023-02-08 23:53:58,846 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x23b09932566600c4: from storage DS-0dbaaba5-6e2a-4661-a4e7-d380caea9a6f node DatanodeRegistration(127.0.0.1:35303, datanodeUuid=e53af517-ccea-4786-a9f1-d1d57e9a860e, infoPort=45991, infoSecurePort=0, ipcPort=33685, storageInfo=lv=-57;cid=testClusterID;nsid=1459456370;c=1675900436765), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 23:53:58,846 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6cb358a208cfdeba: Processing first storage report for DS-d2f39bb4-4fc9-47ac-a69f-f7fba1270d0c from datanode c96d9a8d-7822-404e-9335-ec30f9bdbf87 2023-02-08 23:53:58,846 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6cb358a208cfdeba: from storage DS-d2f39bb4-4fc9-47ac-a69f-f7fba1270d0c node DatanodeRegistration(127.0.0.1:45117, datanodeUuid=c96d9a8d-7822-404e-9335-ec30f9bdbf87, infoPort=34387, infoSecurePort=0, ipcPort=37973, storageInfo=lv=-57;cid=testClusterID;nsid=1459456370;c=1675900436765), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 23:53:58,847 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x12f85878f61cda94: Processing first storage report for DS-27133a68-6054-483f-a193-1b0e576ddc55 from datanode d39c9230-801b-44ca-bfae-38ddf4db7d5a 2023-02-08 23:53:58,847 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x12f85878f61cda94: from storage DS-27133a68-6054-483f-a193-1b0e576ddc55 node DatanodeRegistration(127.0.0.1:40815, datanodeUuid=d39c9230-801b-44ca-bfae-38ddf4db7d5a, infoPort=44023, infoSecurePort=0, ipcPort=39373, storageInfo=lv=-57;cid=testClusterID;nsid=1459456370;c=1675900436765), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 23:53:58,847 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x23b09932566600c4: Processing first storage report for DS-4c1a81d1-ae86-4d2c-8754-510186872fbf from datanode e53af517-ccea-4786-a9f1-d1d57e9a860e 2023-02-08 23:53:58,847 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x23b09932566600c4: from storage DS-4c1a81d1-ae86-4d2c-8754-510186872fbf node DatanodeRegistration(127.0.0.1:35303, datanodeUuid=e53af517-ccea-4786-a9f1-d1d57e9a860e, infoPort=45991, infoSecurePort=0, ipcPort=33685, storageInfo=lv=-57;cid=testClusterID;nsid=1459456370;c=1675900436765), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 23:53:59,029 DEBUG [Listener at localhost/37973] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d 2023-02-08 23:53:59,091 INFO [Listener at localhost/37973] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/zookeeper_0, clientPort=51518, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-08 23:53:59,105 INFO [Listener at localhost/37973] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=51518 2023-02-08 23:53:59,115 INFO [Listener at localhost/37973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:53:59,118 INFO [Listener at localhost/37973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:53:59,780 INFO [Listener at localhost/37973] util.FSUtils(479): Created version file at hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed with version=8 2023-02-08 23:53:59,780 INFO [Listener at localhost/37973] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/hbase-staging 2023-02-08 23:54:00,079 INFO [Listener at localhost/37973] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-08 23:54:00,546 INFO [Listener at localhost/37973] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-08 23:54:00,577 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 23:54:00,578 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 23:54:00,578 INFO [Listener at localhost/37973] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 23:54:00,578 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 23:54:00,578 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 23:54:00,716 INFO [Listener at localhost/37973] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 23:54:00,787 DEBUG [Listener at localhost/37973] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-08 23:54:00,890 INFO [Listener at localhost/37973] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37353 2023-02-08 23:54:00,900 INFO [Listener at localhost/37973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:54:00,902 INFO [Listener at localhost/37973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:54:00,922 INFO [Listener at localhost/37973] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37353 connecting to ZooKeeper ensemble=127.0.0.1:51518 2023-02-08 23:54:00,961 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:373530x0, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 23:54:00,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:37353-0x100365b33e90000 connected 2023-02-08 23:54:00,986 DEBUG [Listener at localhost/37973] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 23:54:00,987 DEBUG [Listener at localhost/37973] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 23:54:00,991 DEBUG [Listener at localhost/37973] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 23:54:00,999 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37353 2023-02-08 23:54:00,999 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37353 2023-02-08 23:54:00,999 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37353 2023-02-08 23:54:01,000 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37353 2023-02-08 23:54:01,001 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37353 2023-02-08 23:54:01,006 INFO [Listener at localhost/37973] master.HMaster(439): hbase.rootdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed, hbase.cluster.distributed=false 2023-02-08 23:54:01,075 INFO [Listener at localhost/37973] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-08 23:54:01,076 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 23:54:01,076 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 23:54:01,076 INFO [Listener at localhost/37973] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 23:54:01,076 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 23:54:01,077 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 23:54:01,081 INFO [Listener at localhost/37973] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 23:54:01,084 INFO [Listener at localhost/37973] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36713 2023-02-08 23:54:01,087 INFO [Listener at localhost/37973] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-08 23:54:01,093 DEBUG [Listener at localhost/37973] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-08 23:54:01,095 INFO [Listener at localhost/37973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:54:01,097 INFO [Listener at localhost/37973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:54:01,098 INFO [Listener at localhost/37973] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36713 connecting to ZooKeeper ensemble=127.0.0.1:51518 2023-02-08 23:54:01,103 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:367130x0, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 23:54:01,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:36713-0x100365b33e90001 connected 2023-02-08 23:54:01,104 DEBUG [Listener at localhost/37973] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 23:54:01,105 DEBUG [Listener at localhost/37973] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 23:54:01,106 DEBUG [Listener at localhost/37973] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 23:54:01,107 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36713 2023-02-08 23:54:01,107 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36713 2023-02-08 23:54:01,107 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36713 2023-02-08 23:54:01,108 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36713 2023-02-08 23:54:01,108 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36713 2023-02-08 23:54:01,121 INFO [Listener at localhost/37973] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-08 23:54:01,121 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 23:54:01,121 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 23:54:01,121 INFO [Listener at localhost/37973] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 23:54:01,121 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 23:54:01,122 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 23:54:01,122 INFO [Listener at localhost/37973] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 23:54:01,123 INFO [Listener at localhost/37973] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36199 2023-02-08 23:54:01,124 INFO [Listener at localhost/37973] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-08 23:54:01,125 DEBUG [Listener at localhost/37973] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-08 23:54:01,126 INFO [Listener at localhost/37973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:54:01,128 INFO [Listener at localhost/37973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:54:01,131 INFO [Listener at localhost/37973] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36199 connecting to ZooKeeper ensemble=127.0.0.1:51518 2023-02-08 23:54:01,135 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:361990x0, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 23:54:01,136 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:36199-0x100365b33e90002 connected 2023-02-08 23:54:01,136 DEBUG [Listener at localhost/37973] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 23:54:01,137 DEBUG [Listener at localhost/37973] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 23:54:01,138 DEBUG [Listener at localhost/37973] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 23:54:01,138 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36199 2023-02-08 23:54:01,139 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36199 2023-02-08 23:54:01,139 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36199 2023-02-08 23:54:01,139 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36199 2023-02-08 23:54:01,140 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36199 2023-02-08 23:54:01,152 INFO [Listener at localhost/37973] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-08 23:54:01,152 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 23:54:01,152 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 23:54:01,153 INFO [Listener at localhost/37973] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 23:54:01,153 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 23:54:01,153 INFO [Listener at localhost/37973] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 23:54:01,153 INFO [Listener at localhost/37973] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 23:54:01,156 INFO [Listener at localhost/37973] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42343 2023-02-08 23:54:01,156 INFO [Listener at localhost/37973] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-08 23:54:01,157 DEBUG [Listener at localhost/37973] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-08 23:54:01,158 INFO [Listener at localhost/37973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:54:01,160 INFO [Listener at localhost/37973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:54:01,161 INFO [Listener at localhost/37973] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42343 connecting to ZooKeeper ensemble=127.0.0.1:51518 2023-02-08 23:54:01,165 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:423430x0, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 23:54:01,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:42343-0x100365b33e90003 connected 2023-02-08 23:54:01,166 DEBUG [Listener at localhost/37973] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 23:54:01,166 DEBUG [Listener at localhost/37973] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 23:54:01,167 DEBUG [Listener at localhost/37973] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 23:54:01,167 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42343 2023-02-08 23:54:01,168 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42343 2023-02-08 23:54:01,168 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42343 2023-02-08 23:54:01,168 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42343 2023-02-08 23:54:01,169 DEBUG [Listener at localhost/37973] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42343 2023-02-08 23:54:01,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37353,1675900439913 2023-02-08 23:54:01,180 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-08 23:54:01,181 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37353,1675900439913 2023-02-08 23:54:01,199 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 23:54:01,199 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 23:54:01,199 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 23:54:01,199 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 23:54:01,200 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:54:01,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-08 23:54:01,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37353,1675900439913 from backup master directory 2023-02-08 23:54:01,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-08 23:54:01,205 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37353,1675900439913 2023-02-08 23:54:01,206 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-08 23:54:01,206 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 23:54:01,206 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,37353,1675900439913 2023-02-08 23:54:01,209 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-08 23:54:01,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-08 23:54:01,301 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/hbase.id with ID: 62637dd0-fa31-46ac-87e7-2a5fc2ef9c34 2023-02-08 23:54:01,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:54:01,359 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:54:01,409 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x585d291e to 127.0.0.1:51518 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:01,442 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@35e82cd7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:01,465 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-08 23:54:01,467 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-08 23:54:01,484 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-08 23:54:01,484 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-08 23:54:01,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-08 23:54:01,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-08 23:54:01,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 23:54:01,527 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/data/master/store-tmp 2023-02-08 23:54:01,575 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:01,575 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-08 23:54:01,576 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:54:01,576 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:54:01,576 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-08 23:54:01,576 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:54:01,576 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:54:01,576 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675900441575Disabling compacts and flushes for region at 1675900441575Disabling writes for close at 1675900441576 (+1 ms)Writing region close event to WAL at 1675900441576Closed at 1675900441576 2023-02-08 23:54:01,578 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/WALs/jenkins-hbase4.apache.org,37353,1675900439913 2023-02-08 23:54:01,598 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37353%2C1675900439913, suffix=, logDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/WALs/jenkins-hbase4.apache.org,37353,1675900439913, archiveDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/oldWALs, maxLogs=10 2023-02-08 23:54:01,651 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45117,DS-78636c1c-1e4c-456a-9f65-6d5b741dfa42,DISK] 2023-02-08 23:54:01,651 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35303,DS-0dbaaba5-6e2a-4661-a4e7-d380caea9a6f,DISK] 2023-02-08 23:54:01,651 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40815,DS-a61b8f87-3dea-401a-b95a-91f3efd93fae,DISK] 2023-02-08 23:54:01,659 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-08 23:54:01,724 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/WALs/jenkins-hbase4.apache.org,37353,1675900439913/jenkins-hbase4.apache.org%2C37353%2C1675900439913.1675900441608 2023-02-08 23:54:01,725 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45117,DS-78636c1c-1e4c-456a-9f65-6d5b741dfa42,DISK], DatanodeInfoWithStorage[127.0.0.1:40815,DS-a61b8f87-3dea-401a-b95a-91f3efd93fae,DISK], DatanodeInfoWithStorage[127.0.0.1:35303,DS-0dbaaba5-6e2a-4661-a4e7-d380caea9a6f,DISK]] 2023-02-08 23:54:01,726 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:01,726 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:01,729 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 23:54:01,730 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 23:54:01,786 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-08 23:54:01,793 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-08 23:54:01,817 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-08 23:54:01,829 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:01,835 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-08 23:54:01,837 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-08 23:54:01,850 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 23:54:01,854 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:01,855 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71840187, jitterRate=0.07050220668315887}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:54:01,856 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675900441755Initializing all the Stores at 1675900441757 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900441758 (+1 ms)Cleaning up temporary data from old regions at 1675900441843 (+85 ms)Cleaning up detritus from prior splits at 1675900441844 (+1 ms)Region opened successfully at 1675900441856 (+12 ms) 2023-02-08 23:54:01,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-08 23:54:01,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:01,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:01,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:01,883 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-08 23:54:01,919 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 36 msec 2023-02-08 23:54:01,920 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-08 23:54:01,943 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-08 23:54:01,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-08 23:54:01,972 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-08 23:54:01,975 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-08 23:54:01,977 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-08 23:54:01,982 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-08 23:54:01,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-08 23:54:01,988 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:54:01,990 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-08 23:54:01,990 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-08 23:54:02,001 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-08 23:54:02,006 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 23:54:02,006 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 23:54:02,006 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 23:54:02,006 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 23:54:02,007 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:54:02,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,37353,1675900439913, sessionid=0x100365b33e90000, setting cluster-up flag (Was=false) 2023-02-08 23:54:02,030 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:54:02,035 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-08 23:54:02,036 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37353,1675900439913 2023-02-08 23:54:02,045 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:54:02,050 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-08 23:54:02,051 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37353,1675900439913 2023-02-08 23:54:02,054 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.hbase-snapshot/.tmp 2023-02-08 23:54:02,073 INFO [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(952): ClusterId : 62637dd0-fa31-46ac-87e7-2a5fc2ef9c34 2023-02-08 23:54:02,073 INFO [RS:0;jenkins-hbase4:36713] regionserver.HRegionServer(952): ClusterId : 62637dd0-fa31-46ac-87e7-2a5fc2ef9c34 2023-02-08 23:54:02,073 INFO [RS:2;jenkins-hbase4:42343] regionserver.HRegionServer(952): ClusterId : 62637dd0-fa31-46ac-87e7-2a5fc2ef9c34 2023-02-08 23:54:02,079 DEBUG [RS:2;jenkins-hbase4:42343] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-08 23:54:02,079 DEBUG [RS:1;jenkins-hbase4:36199] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-08 23:54:02,079 DEBUG [RS:0;jenkins-hbase4:36713] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-08 23:54:02,085 DEBUG [RS:1;jenkins-hbase4:36199] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-08 23:54:02,085 DEBUG [RS:2;jenkins-hbase4:42343] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-08 23:54:02,085 DEBUG [RS:0;jenkins-hbase4:36713] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-08 23:54:02,085 DEBUG [RS:2;jenkins-hbase4:42343] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-08 23:54:02,085 DEBUG [RS:1;jenkins-hbase4:36199] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-08 23:54:02,085 DEBUG [RS:0;jenkins-hbase4:36713] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-08 23:54:02,090 DEBUG [RS:1;jenkins-hbase4:36199] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-08 23:54:02,090 DEBUG [RS:2;jenkins-hbase4:42343] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-08 23:54:02,090 DEBUG [RS:0;jenkins-hbase4:36713] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-08 23:54:02,092 DEBUG [RS:2;jenkins-hbase4:42343] zookeeper.ReadOnlyZKClient(139): Connect 0x5f60e7a6 to 127.0.0.1:51518 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:02,092 DEBUG [RS:0;jenkins-hbase4:36713] zookeeper.ReadOnlyZKClient(139): Connect 0x27ae63d4 to 127.0.0.1:51518 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:02,092 DEBUG [RS:1;jenkins-hbase4:36199] zookeeper.ReadOnlyZKClient(139): Connect 0x0a9c82ad to 127.0.0.1:51518 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:02,106 DEBUG [RS:0;jenkins-hbase4:36713] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@29699951, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:02,106 DEBUG [RS:0;jenkins-hbase4:36713] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d5608a4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-08 23:54:02,108 DEBUG [RS:2;jenkins-hbase4:42343] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@51a7123a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:02,108 DEBUG [RS:1;jenkins-hbase4:36199] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@45b7d8e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:02,108 DEBUG [RS:2;jenkins-hbase4:42343] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@55988cd6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-08 23:54:02,108 DEBUG [RS:1;jenkins-hbase4:36199] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@64e9dd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-08 23:54:02,134 DEBUG [RS:1;jenkins-hbase4:36199] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:36199 2023-02-08 23:54:02,136 DEBUG [RS:2;jenkins-hbase4:42343] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:42343 2023-02-08 23:54:02,138 DEBUG [RS:0;jenkins-hbase4:36713] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36713 2023-02-08 23:54:02,139 INFO [RS:2;jenkins-hbase4:42343] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-08 23:54:02,139 INFO [RS:1;jenkins-hbase4:36199] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-08 23:54:02,139 INFO [RS:2;jenkins-hbase4:42343] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-08 23:54:02,139 INFO [RS:0;jenkins-hbase4:36713] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-08 23:54:02,139 DEBUG [RS:2;jenkins-hbase4:42343] regionserver.HRegionServer(1023): About to register with Master. 2023-02-08 23:54:02,139 INFO [RS:1;jenkins-hbase4:36199] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-08 23:54:02,139 INFO [RS:0;jenkins-hbase4:36713] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-08 23:54:02,139 DEBUG [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1023): About to register with Master. 2023-02-08 23:54:02,140 DEBUG [RS:0;jenkins-hbase4:36713] regionserver.HRegionServer(1023): About to register with Master. 2023-02-08 23:54:02,142 INFO [RS:2;jenkins-hbase4:42343] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37353,1675900439913 with isa=jenkins-hbase4.apache.org/172.31.14.131:42343, startcode=1675900441151 2023-02-08 23:54:02,142 INFO [RS:0;jenkins-hbase4:36713] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37353,1675900439913 with isa=jenkins-hbase4.apache.org/172.31.14.131:36713, startcode=1675900441074 2023-02-08 23:54:02,142 INFO [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37353,1675900439913 with isa=jenkins-hbase4.apache.org/172.31.14.131:36199, startcode=1675900441120 2023-02-08 23:54:02,162 DEBUG [RS:0;jenkins-hbase4:36713] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-08 23:54:02,162 DEBUG [RS:2;jenkins-hbase4:42343] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-08 23:54:02,162 DEBUG [RS:1;jenkins-hbase4:36199] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-08 23:54:02,183 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:02,196 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 23:54:02,197 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 23:54:02,197 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 23:54:02,197 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 23:54:02,197 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-08 23:54:02,197 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,197 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-08 23:54:02,197 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900472198 2023-02-08 23:54:02,200 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-08 23:54:02,204 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:02,205 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-08 23:54:02,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-08 23:54:02,217 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-08 23:54:02,224 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-08 23:54:02,225 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-08 23:54:02,225 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-08 23:54:02,225 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-08 23:54:02,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,226 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55869, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-08 23:54:02,227 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44641, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-08 23:54:02,227 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54865, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-08 23:54:02,229 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-08 23:54:02,231 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-08 23:54:02,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-08 23:54:02,239 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-08 23:54:02,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-08 23:54:02,242 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675900442242,5,FailOnTimeoutGroup] 2023-02-08 23:54:02,243 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675900442242,5,FailOnTimeoutGroup] 2023-02-08 23:54:02,243 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,243 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-08 23:54:02,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,245 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,257 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37353] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:02,259 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37353] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:02,260 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:02,274 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:02,275 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:02,275 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed 2023-02-08 23:54:02,285 DEBUG [RS:2;jenkins-hbase4:42343] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed 2023-02-08 23:54:02,285 DEBUG [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed 2023-02-08 23:54:02,285 DEBUG [RS:0;jenkins-hbase4:36713] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed 2023-02-08 23:54:02,286 DEBUG [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38079 2023-02-08 23:54:02,285 DEBUG [RS:2;jenkins-hbase4:42343] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38079 2023-02-08 23:54:02,286 DEBUG [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-08 23:54:02,286 DEBUG [RS:0;jenkins-hbase4:36713] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38079 2023-02-08 23:54:02,286 DEBUG [RS:2;jenkins-hbase4:42343] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-08 23:54:02,286 DEBUG [RS:0;jenkins-hbase4:36713] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-08 23:54:02,297 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 23:54:02,300 DEBUG [RS:0;jenkins-hbase4:36713] zookeeper.ZKUtil(162): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:02,300 DEBUG [RS:2;jenkins-hbase4:42343] zookeeper.ZKUtil(162): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:02,300 DEBUG [RS:1;jenkins-hbase4:36199] zookeeper.ZKUtil(162): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:02,300 WARN [RS:0;jenkins-hbase4:36713] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 23:54:02,301 WARN [RS:2;jenkins-hbase4:42343] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 23:54:02,301 WARN [RS:1;jenkins-hbase4:36199] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 23:54:02,302 INFO [RS:0;jenkins-hbase4:36713] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 23:54:02,302 DEBUG [RS:0;jenkins-hbase4:36713] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/WALs/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:02,303 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42343,1675900441151] 2023-02-08 23:54:02,303 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36713,1675900441074] 2023-02-08 23:54:02,303 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36199,1675900441120] 2023-02-08 23:54:02,302 INFO [RS:1;jenkins-hbase4:36199] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 23:54:02,302 INFO [RS:2;jenkins-hbase4:42343] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 23:54:02,303 DEBUG [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/WALs/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:02,303 DEBUG [RS:2;jenkins-hbase4:42343] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/WALs/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:02,315 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:02,317 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-08 23:54:02,319 DEBUG [RS:1;jenkins-hbase4:36199] zookeeper.ZKUtil(162): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:02,319 DEBUG [RS:2;jenkins-hbase4:42343] zookeeper.ZKUtil(162): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:02,319 DEBUG [RS:0;jenkins-hbase4:36713] zookeeper.ZKUtil(162): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:02,320 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/info 2023-02-08 23:54:02,320 DEBUG [RS:1;jenkins-hbase4:36199] zookeeper.ZKUtil(162): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:02,320 DEBUG [RS:2;jenkins-hbase4:42343] zookeeper.ZKUtil(162): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:02,320 DEBUG [RS:0;jenkins-hbase4:36713] zookeeper.ZKUtil(162): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:02,321 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-08 23:54:02,321 DEBUG [RS:2;jenkins-hbase4:42343] zookeeper.ZKUtil(162): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:02,321 DEBUG [RS:0;jenkins-hbase4:36713] zookeeper.ZKUtil(162): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:02,321 DEBUG [RS:1;jenkins-hbase4:36199] zookeeper.ZKUtil(162): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:02,322 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:02,322 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-08 23:54:02,325 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/rep_barrier 2023-02-08 23:54:02,326 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-08 23:54:02,326 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:02,327 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-08 23:54:02,329 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/table 2023-02-08 23:54:02,329 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-08 23:54:02,330 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:02,333 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740 2023-02-08 23:54:02,333 DEBUG [RS:0;jenkins-hbase4:36713] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-08 23:54:02,333 DEBUG [RS:2;jenkins-hbase4:42343] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-08 23:54:02,333 DEBUG [RS:1;jenkins-hbase4:36199] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-08 23:54:02,334 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740 2023-02-08 23:54:02,339 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-08 23:54:02,341 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-08 23:54:02,344 INFO [RS:1;jenkins-hbase4:36199] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-08 23:54:02,344 INFO [RS:0;jenkins-hbase4:36713] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-08 23:54:02,344 INFO [RS:2;jenkins-hbase4:42343] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-08 23:54:02,346 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:02,347 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69371410, jitterRate=0.03371456265449524}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-08 23:54:02,347 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675900442315Initializing all the Stores at 1675900442316 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675900442316Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900442317 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675900442317Cleaning up temporary data from old regions at 1675900442336 (+19 ms)Cleaning up detritus from prior splits at 1675900442337 (+1 ms)Region opened successfully at 1675900442347 (+10 ms) 2023-02-08 23:54:02,347 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-08 23:54:02,347 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-08 23:54:02,347 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-08 23:54:02,347 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-08 23:54:02,347 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-08 23:54:02,349 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-08 23:54:02,349 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675900442347Disabling compacts and flushes for region at 1675900442347Disabling writes for close at 1675900442347Writing region close event to WAL at 1675900442349 (+2 ms)Closed at 1675900442349 2023-02-08 23:54:02,369 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:02,369 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-08 23:54:02,378 INFO [RS:0;jenkins-hbase4:36713] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-08 23:54:02,378 INFO [RS:1;jenkins-hbase4:36199] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-08 23:54:02,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-08 23:54:02,380 INFO [RS:2;jenkins-hbase4:42343] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-08 23:54:02,390 INFO [RS:0;jenkins-hbase4:36713] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:54:02,390 INFO [RS:0;jenkins-hbase4:36713] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,397 INFO [RS:1;jenkins-hbase4:36199] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:54:02,397 INFO [RS:1;jenkins-hbase4:36199] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,393 INFO [RS:2;jenkins-hbase4:42343] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:54:02,397 INFO [RS:2;jenkins-hbase4:42343] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,404 INFO [RS:0;jenkins-hbase4:36713] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-08 23:54:02,404 INFO [RS:2;jenkins-hbase4:42343] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-08 23:54:02,409 INFO [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-08 23:54:02,410 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-08 23:54:02,413 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-08 23:54:02,420 INFO [RS:1;jenkins-hbase4:36199] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,420 INFO [RS:0;jenkins-hbase4:36713] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,420 INFO [RS:2;jenkins-hbase4:42343] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,420 DEBUG [RS:1;jenkins-hbase4:36199] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,420 DEBUG [RS:0;jenkins-hbase4:36713] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,421 DEBUG [RS:1;jenkins-hbase4:36199] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,421 DEBUG [RS:0;jenkins-hbase4:36713] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,421 DEBUG [RS:1;jenkins-hbase4:36199] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,421 DEBUG [RS:0;jenkins-hbase4:36713] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,421 DEBUG [RS:2;jenkins-hbase4:42343] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,421 DEBUG [RS:0;jenkins-hbase4:36713] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,421 DEBUG [RS:2;jenkins-hbase4:42343] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,421 DEBUG [RS:0;jenkins-hbase4:36713] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,421 DEBUG [RS:2;jenkins-hbase4:42343] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,421 DEBUG [RS:1;jenkins-hbase4:36199] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,422 DEBUG [RS:2;jenkins-hbase4:42343] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,422 DEBUG [RS:0;jenkins-hbase4:36713] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-08 23:54:02,422 DEBUG [RS:2;jenkins-hbase4:42343] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,422 DEBUG [RS:0;jenkins-hbase4:36713] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,422 DEBUG [RS:1;jenkins-hbase4:36199] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,422 DEBUG [RS:2;jenkins-hbase4:42343] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-08 23:54:02,422 DEBUG [RS:1;jenkins-hbase4:36199] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-08 23:54:02,422 DEBUG [RS:2;jenkins-hbase4:42343] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,422 DEBUG [RS:0;jenkins-hbase4:36713] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,422 DEBUG [RS:2;jenkins-hbase4:42343] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,422 DEBUG [RS:0;jenkins-hbase4:36713] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,422 DEBUG [RS:1;jenkins-hbase4:36199] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,423 DEBUG [RS:0;jenkins-hbase4:36713] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,423 DEBUG [RS:1;jenkins-hbase4:36199] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,422 DEBUG [RS:2;jenkins-hbase4:42343] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,423 DEBUG [RS:1;jenkins-hbase4:36199] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,423 DEBUG [RS:2;jenkins-hbase4:42343] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,423 DEBUG [RS:1;jenkins-hbase4:36199] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:54:02,437 INFO [RS:0;jenkins-hbase4:36713] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,437 INFO [RS:2;jenkins-hbase4:42343] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,437 INFO [RS:0;jenkins-hbase4:36713] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,437 INFO [RS:2;jenkins-hbase4:42343] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,437 INFO [RS:0;jenkins-hbase4:36713] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,437 INFO [RS:2;jenkins-hbase4:42343] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,437 INFO [RS:1;jenkins-hbase4:36199] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,438 INFO [RS:1;jenkins-hbase4:36199] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,438 INFO [RS:1;jenkins-hbase4:36199] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,455 INFO [RS:1;jenkins-hbase4:36199] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-08 23:54:02,455 INFO [RS:0;jenkins-hbase4:36713] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-08 23:54:02,455 INFO [RS:2;jenkins-hbase4:42343] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-08 23:54:02,459 INFO [RS:1;jenkins-hbase4:36199] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36199,1675900441120-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,459 INFO [RS:2;jenkins-hbase4:42343] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42343,1675900441151-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,459 INFO [RS:0;jenkins-hbase4:36713] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36713,1675900441074-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:02,480 INFO [RS:1;jenkins-hbase4:36199] regionserver.Replication(203): jenkins-hbase4.apache.org,36199,1675900441120 started 2023-02-08 23:54:02,480 INFO [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,36199,1675900441120, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36199, sessionid=0x100365b33e90002 2023-02-08 23:54:02,480 DEBUG [RS:1;jenkins-hbase4:36199] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-08 23:54:02,480 DEBUG [RS:1;jenkins-hbase4:36199] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:02,480 DEBUG [RS:1;jenkins-hbase4:36199] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36199,1675900441120' 2023-02-08 23:54:02,480 DEBUG [RS:1;jenkins-hbase4:36199] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:02,481 DEBUG [RS:1;jenkins-hbase4:36199] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:02,482 DEBUG [RS:1;jenkins-hbase4:36199] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-08 23:54:02,482 DEBUG [RS:1;jenkins-hbase4:36199] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-08 23:54:02,482 DEBUG [RS:1;jenkins-hbase4:36199] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:02,482 DEBUG [RS:1;jenkins-hbase4:36199] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36199,1675900441120' 2023-02-08 23:54:02,482 DEBUG [RS:1;jenkins-hbase4:36199] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 23:54:02,483 DEBUG [RS:1;jenkins-hbase4:36199] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 23:54:02,483 DEBUG [RS:1;jenkins-hbase4:36199] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-08 23:54:02,483 INFO [RS:1;jenkins-hbase4:36199] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-08 23:54:02,483 INFO [RS:1;jenkins-hbase4:36199] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-08 23:54:02,484 INFO [RS:2;jenkins-hbase4:42343] regionserver.Replication(203): jenkins-hbase4.apache.org,42343,1675900441151 started 2023-02-08 23:54:02,484 INFO [RS:2;jenkins-hbase4:42343] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,42343,1675900441151, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42343, sessionid=0x100365b33e90003 2023-02-08 23:54:02,488 INFO [RS:0;jenkins-hbase4:36713] regionserver.Replication(203): jenkins-hbase4.apache.org,36713,1675900441074 started 2023-02-08 23:54:02,488 DEBUG [RS:2;jenkins-hbase4:42343] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-08 23:54:02,488 INFO [RS:0;jenkins-hbase4:36713] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,36713,1675900441074, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36713, sessionid=0x100365b33e90001 2023-02-08 23:54:02,488 DEBUG [RS:2;jenkins-hbase4:42343] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:02,489 DEBUG [RS:0;jenkins-hbase4:36713] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-08 23:54:02,489 DEBUG [RS:0;jenkins-hbase4:36713] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:02,489 DEBUG [RS:0;jenkins-hbase4:36713] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36713,1675900441074' 2023-02-08 23:54:02,489 DEBUG [RS:0;jenkins-hbase4:36713] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:02,489 DEBUG [RS:2;jenkins-hbase4:42343] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42343,1675900441151' 2023-02-08 23:54:02,489 DEBUG [RS:2;jenkins-hbase4:42343] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:02,490 DEBUG [RS:2;jenkins-hbase4:42343] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:02,490 DEBUG [RS:0;jenkins-hbase4:36713] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:02,490 DEBUG [RS:2;jenkins-hbase4:42343] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-08 23:54:02,490 DEBUG [RS:0;jenkins-hbase4:36713] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-08 23:54:02,490 DEBUG [RS:0;jenkins-hbase4:36713] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-08 23:54:02,490 DEBUG [RS:2;jenkins-hbase4:42343] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-08 23:54:02,491 DEBUG [RS:0;jenkins-hbase4:36713] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:02,491 DEBUG [RS:0;jenkins-hbase4:36713] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36713,1675900441074' 2023-02-08 23:54:02,491 DEBUG [RS:0;jenkins-hbase4:36713] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 23:54:02,491 DEBUG [RS:2;jenkins-hbase4:42343] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:02,491 DEBUG [RS:2;jenkins-hbase4:42343] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42343,1675900441151' 2023-02-08 23:54:02,491 DEBUG [RS:2;jenkins-hbase4:42343] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 23:54:02,492 DEBUG [RS:0;jenkins-hbase4:36713] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 23:54:02,492 DEBUG [RS:2;jenkins-hbase4:42343] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 23:54:02,492 DEBUG [RS:0;jenkins-hbase4:36713] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-08 23:54:02,492 INFO [RS:0;jenkins-hbase4:36713] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-08 23:54:02,493 INFO [RS:0;jenkins-hbase4:36713] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-08 23:54:02,494 DEBUG [RS:2;jenkins-hbase4:42343] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-08 23:54:02,494 INFO [RS:2;jenkins-hbase4:42343] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-08 23:54:02,495 INFO [RS:2;jenkins-hbase4:42343] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-08 23:54:02,565 DEBUG [jenkins-hbase4:37353] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-08 23:54:02,569 DEBUG [jenkins-hbase4:37353] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:54:02,575 DEBUG [jenkins-hbase4:37353] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:54:02,575 DEBUG [jenkins-hbase4:37353] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:54:02,575 DEBUG [jenkins-hbase4:37353] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:54:02,575 DEBUG [jenkins-hbase4:37353] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 23:54:02,578 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36199,1675900441120, state=OPENING 2023-02-08 23:54:02,585 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-08 23:54:02,587 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:54:02,588 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-08 23:54:02,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:02,594 INFO [RS:1;jenkins-hbase4:36199] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36199%2C1675900441120, suffix=, logDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/WALs/jenkins-hbase4.apache.org,36199,1675900441120, archiveDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/oldWALs, maxLogs=32 2023-02-08 23:54:02,597 INFO [RS:0;jenkins-hbase4:36713] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36713%2C1675900441074, suffix=, logDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/WALs/jenkins-hbase4.apache.org,36713,1675900441074, archiveDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/oldWALs, maxLogs=32 2023-02-08 23:54:02,600 INFO [RS:2;jenkins-hbase4:42343] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42343%2C1675900441151, suffix=, logDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/WALs/jenkins-hbase4.apache.org,42343,1675900441151, archiveDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/oldWALs, maxLogs=32 2023-02-08 23:54:02,618 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40815,DS-a61b8f87-3dea-401a-b95a-91f3efd93fae,DISK] 2023-02-08 23:54:02,618 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45117,DS-78636c1c-1e4c-456a-9f65-6d5b741dfa42,DISK] 2023-02-08 23:54:02,618 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35303,DS-0dbaaba5-6e2a-4661-a4e7-d380caea9a6f,DISK] 2023-02-08 23:54:02,627 INFO [RS:1;jenkins-hbase4:36199] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/WALs/jenkins-hbase4.apache.org,36199,1675900441120/jenkins-hbase4.apache.org%2C36199%2C1675900441120.1675900442596 2023-02-08 23:54:02,628 DEBUG [RS:1;jenkins-hbase4:36199] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35303,DS-0dbaaba5-6e2a-4661-a4e7-d380caea9a6f,DISK], DatanodeInfoWithStorage[127.0.0.1:40815,DS-a61b8f87-3dea-401a-b95a-91f3efd93fae,DISK], DatanodeInfoWithStorage[127.0.0.1:45117,DS-78636c1c-1e4c-456a-9f65-6d5b741dfa42,DISK]] 2023-02-08 23:54:02,639 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40815,DS-a61b8f87-3dea-401a-b95a-91f3efd93fae,DISK] 2023-02-08 23:54:02,641 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45117,DS-78636c1c-1e4c-456a-9f65-6d5b741dfa42,DISK] 2023-02-08 23:54:02,647 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35303,DS-0dbaaba5-6e2a-4661-a4e7-d380caea9a6f,DISK] 2023-02-08 23:54:02,648 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35303,DS-0dbaaba5-6e2a-4661-a4e7-d380caea9a6f,DISK] 2023-02-08 23:54:02,649 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40815,DS-a61b8f87-3dea-401a-b95a-91f3efd93fae,DISK] 2023-02-08 23:54:02,649 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45117,DS-78636c1c-1e4c-456a-9f65-6d5b741dfa42,DISK] 2023-02-08 23:54:02,662 INFO [RS:2;jenkins-hbase4:42343] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/WALs/jenkins-hbase4.apache.org,42343,1675900441151/jenkins-hbase4.apache.org%2C42343%2C1675900441151.1675900442602 2023-02-08 23:54:02,668 DEBUG [RS:2;jenkins-hbase4:42343] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45117,DS-78636c1c-1e4c-456a-9f65-6d5b741dfa42,DISK], DatanodeInfoWithStorage[127.0.0.1:35303,DS-0dbaaba5-6e2a-4661-a4e7-d380caea9a6f,DISK], DatanodeInfoWithStorage[127.0.0.1:40815,DS-a61b8f87-3dea-401a-b95a-91f3efd93fae,DISK]] 2023-02-08 23:54:02,671 INFO [RS:0;jenkins-hbase4:36713] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/WALs/jenkins-hbase4.apache.org,36713,1675900441074/jenkins-hbase4.apache.org%2C36713%2C1675900441074.1675900442599 2023-02-08 23:54:02,672 DEBUG [RS:0;jenkins-hbase4:36713] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35303,DS-0dbaaba5-6e2a-4661-a4e7-d380caea9a6f,DISK], DatanodeInfoWithStorage[127.0.0.1:40815,DS-a61b8f87-3dea-401a-b95a-91f3efd93fae,DISK], DatanodeInfoWithStorage[127.0.0.1:45117,DS-78636c1c-1e4c-456a-9f65-6d5b741dfa42,DISK]] 2023-02-08 23:54:02,782 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:02,785 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 23:54:02,788 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38656, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 23:54:02,802 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-08 23:54:02,802 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 23:54:02,806 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36199%2C1675900441120.meta, suffix=.meta, logDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/WALs/jenkins-hbase4.apache.org,36199,1675900441120, archiveDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/oldWALs, maxLogs=32 2023-02-08 23:54:02,828 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40815,DS-a61b8f87-3dea-401a-b95a-91f3efd93fae,DISK] 2023-02-08 23:54:02,828 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45117,DS-78636c1c-1e4c-456a-9f65-6d5b741dfa42,DISK] 2023-02-08 23:54:02,830 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35303,DS-0dbaaba5-6e2a-4661-a4e7-d380caea9a6f,DISK] 2023-02-08 23:54:02,839 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/WALs/jenkins-hbase4.apache.org,36199,1675900441120/jenkins-hbase4.apache.org%2C36199%2C1675900441120.meta.1675900442807.meta 2023-02-08 23:54:02,840 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40815,DS-a61b8f87-3dea-401a-b95a-91f3efd93fae,DISK], DatanodeInfoWithStorage[127.0.0.1:45117,DS-78636c1c-1e4c-456a-9f65-6d5b741dfa42,DISK], DatanodeInfoWithStorage[127.0.0.1:35303,DS-0dbaaba5-6e2a-4661-a4e7-d380caea9a6f,DISK]] 2023-02-08 23:54:02,840 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:02,844 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:02,845 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-08 23:54:02,863 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-08 23:54:02,866 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-08 23:54:02,871 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-08 23:54:02,871 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:02,871 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-08 23:54:02,871 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-08 23:54:02,874 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-08 23:54:02,877 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/info 2023-02-08 23:54:02,877 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/info 2023-02-08 23:54:02,878 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-08 23:54:02,880 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:02,881 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-08 23:54:02,885 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/rep_barrier 2023-02-08 23:54:02,886 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/rep_barrier 2023-02-08 23:54:02,886 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-08 23:54:02,888 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:02,888 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-08 23:54:02,891 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/table 2023-02-08 23:54:02,891 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/table 2023-02-08 23:54:02,891 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-08 23:54:02,892 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:02,894 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740 2023-02-08 23:54:02,898 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740 2023-02-08 23:54:02,902 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-08 23:54:02,905 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-08 23:54:02,911 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66527775, jitterRate=-0.00865890085697174}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-08 23:54:02,911 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675900442872Writing region info on filesystem at 1675900442872Initializing all the Stores at 1675900442874 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675900442874Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900442874Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675900442874Cleaning up temporary data from old regions at 1675900442900 (+26 ms)Cleaning up detritus from prior splits at 1675900442901 (+1 ms)Running coprocessor post-open hooks at 1675900442911 (+10 ms)Region opened successfully at 1675900442911 2023-02-08 23:54:02,926 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675900442771 2023-02-08 23:54:02,952 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-08 23:54:02,953 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-08 23:54:02,954 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36199,1675900441120, state=OPEN 2023-02-08 23:54:02,958 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-08 23:54:02,959 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-08 23:54:02,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-08 23:54:02,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36199,1675900441120 in 368 msec 2023-02-08 23:54:02,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-08 23:54:02,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 584 msec 2023-02-08 23:54:02,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 868 msec 2023-02-08 23:54:02,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675900441195, completionTime=-1 2023-02-08 23:54:02,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-08 23:54:02,978 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:03,049 DEBUG [hconnection-0x49c176f2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:54:03,052 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38668, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:54:03,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:03,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900503070 2023-02-08 23:54:03,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900563070 2023-02-08 23:54:03,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 92 msec 2023-02-08 23:54:03,100 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37353,1675900439913-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:03,100 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37353,1675900439913-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:03,100 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37353,1675900439913-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:03,102 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37353, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:03,102 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-08 23:54:03,109 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-08 23:54:03,118 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-08 23:54:03,119 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-08 23:54:03,129 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:03,132 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:54:03,134 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:54:03,157 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:54:03,159 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c empty. 2023-02-08 23:54:03,160 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:54:03,160 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-08 23:54:03,198 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:03,202 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 6a82652058ff2f6139e2701c4c52385c, NAME => 'hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp 2023-02-08 23:54:03,223 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:03,223 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 6a82652058ff2f6139e2701c4c52385c, disabling compactions & flushes 2023-02-08 23:54:03,223 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. 2023-02-08 23:54:03,223 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. 2023-02-08 23:54:03,224 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. after waiting 0 ms 2023-02-08 23:54:03,224 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. 2023-02-08 23:54:03,224 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. 2023-02-08 23:54:03,224 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 6a82652058ff2f6139e2701c4c52385c: Waiting for close lock at 1675900443223Disabling compacts and flushes for region at 1675900443223Disabling writes for close at 1675900443224 (+1 ms)Writing region close event to WAL at 1675900443224Closed at 1675900443224 2023-02-08 23:54:03,229 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:54:03,244 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675900443231"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900443231"}]},"ts":"1675900443231"} 2023-02-08 23:54:03,272 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:54:03,274 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:54:03,279 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900443275"}]},"ts":"1675900443275"} 2023-02-08 23:54:03,284 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-08 23:54:03,290 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:54:03,291 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:54:03,291 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:54:03,291 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:54:03,291 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 23:54:03,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6a82652058ff2f6139e2701c4c52385c, ASSIGN}] 2023-02-08 23:54:03,299 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6a82652058ff2f6139e2701c4c52385c, ASSIGN 2023-02-08 23:54:03,301 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=6a82652058ff2f6139e2701c4c52385c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=false 2023-02-08 23:54:03,453 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:54:03,455 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6a82652058ff2f6139e2701c4c52385c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:03,455 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675900443454"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900443454"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900443454"}]},"ts":"1675900443454"} 2023-02-08 23:54:03,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 6a82652058ff2f6139e2701c4c52385c, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:03,624 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. 2023-02-08 23:54:03,624 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6a82652058ff2f6139e2701c4c52385c, NAME => 'hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:03,625 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:03,626 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:54:03,626 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:03,626 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:54:03,626 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:54:03,628 INFO [StoreOpener-6a82652058ff2f6139e2701c4c52385c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:54:03,630 DEBUG [StoreOpener-6a82652058ff2f6139e2701c4c52385c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c/info 2023-02-08 23:54:03,630 DEBUG [StoreOpener-6a82652058ff2f6139e2701c4c52385c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c/info 2023-02-08 23:54:03,631 INFO [StoreOpener-6a82652058ff2f6139e2701c4c52385c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a82652058ff2f6139e2701c4c52385c columnFamilyName info 2023-02-08 23:54:03,632 INFO [StoreOpener-6a82652058ff2f6139e2701c4c52385c-1] regionserver.HStore(310): Store=6a82652058ff2f6139e2701c4c52385c/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:03,634 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:54:03,635 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:54:03,639 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:54:03,642 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:03,643 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6a82652058ff2f6139e2701c4c52385c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62832639, jitterRate=-0.0637207180261612}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:54:03,643 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6a82652058ff2f6139e2701c4c52385c: Running coprocessor pre-open hook at 1675900443626Writing region info on filesystem at 1675900443626Initializing all the Stores at 1675900443628 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675900443628Cleaning up temporary data from old regions at 1675900443635 (+7 ms)Cleaning up detritus from prior splits at 1675900443636 (+1 ms)Running coprocessor post-open hooks at 1675900443643 (+7 ms)Region opened successfully at 1675900443643 2023-02-08 23:54:03,645 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c., pid=6, masterSystemTime=1675900443617 2023-02-08 23:54:03,649 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. 2023-02-08 23:54:03,649 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. 2023-02-08 23:54:03,650 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:03,651 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675900443649"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900443649"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900443649"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900443649"}]},"ts":"1675900443649"} 2023-02-08 23:54:03,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-08 23:54:03,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 6a82652058ff2f6139e2701c4c52385c, server=jenkins-hbase4.apache.org,36199,1675900441120 in 193 msec 2023-02-08 23:54:03,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-08 23:54:03,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=6a82652058ff2f6139e2701c4c52385c, ASSIGN in 363 msec 2023-02-08 23:54:03,664 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:54:03,664 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900443664"}]},"ts":"1675900443664"} 2023-02-08 23:54:03,667 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-08 23:54:03,670 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:54:03,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 549 msec 2023-02-08 23:54:03,732 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-08 23:54:03,734 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-08 23:54:03,734 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:54:03,805 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:03,824 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-08 23:54:03,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 70 msec 2023-02-08 23:54:03,839 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:03,854 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-08 23:54:03,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 21 msec 2023-02-08 23:54:03,881 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-08 23:54:03,884 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-08 23:54:03,885 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.679sec 2023-02-08 23:54:03,889 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-08 23:54:03,891 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-08 23:54:03,891 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-08 23:54:03,893 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37353,1675900439913-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-08 23:54:03,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37353,1675900439913-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-08 23:54:03,903 DEBUG [Listener at localhost/37973] zookeeper.ReadOnlyZKClient(139): Connect 0x328d140f to 127.0.0.1:51518 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:03,907 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-08 23:54:03,908 DEBUG [Listener at localhost/37973] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37e887de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:03,932 DEBUG [hconnection-0x5adffcfe-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:54:03,946 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53688, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:54:03,958 INFO [Listener at localhost/37973] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,37353,1675900439913 2023-02-08 23:54:03,989 INFO [Listener at localhost/37973] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=360, OpenFileDescriptor=696, MaxFileDescriptor=60000, SystemLoadAverage=286, ProcessCount=168, AvailableMemoryMB=3671 2023-02-08 23:54:03,994 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:03,995 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:03,995 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:04,003 DEBUG [Listener at localhost/37973] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:54:04,007 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37384, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:54:04,022 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-08 23:54:04,022 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:54:04,024 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 23:54:04,030 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,033 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:54:04,035 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:54:04,038 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:04,039 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f empty. 2023-02-08 23:54:04,042 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:04,042 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-08 23:54:04,066 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:04,068 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 952532b0193ae630451f659ecaa3180f, NAME => 'testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp 2023-02-08 23:54:04,086 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:04,087 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing 952532b0193ae630451f659ecaa3180f, disabling compactions & flushes 2023-02-08 23:54:04,087 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:04,087 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:04,087 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. after waiting 0 ms 2023-02-08 23:54:04,087 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:04,087 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:04,087 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 952532b0193ae630451f659ecaa3180f: Waiting for close lock at 1675900444087Disabling compacts and flushes for region at 1675900444087Disabling writes for close at 1675900444087Writing region close event to WAL at 1675900444087Closed at 1675900444087 2023-02-08 23:54:04,091 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:54:04,093 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675900444093"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900444093"}]},"ts":"1675900444093"} 2023-02-08 23:54:04,096 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:54:04,097 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:54:04,097 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900444097"}]},"ts":"1675900444097"} 2023-02-08 23:54:04,100 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-08 23:54:04,104 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:54:04,106 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:54:04,106 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:54:04,106 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:54:04,106 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 23:54:04,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=952532b0193ae630451f659ecaa3180f, ASSIGN}] 2023-02-08 23:54:04,109 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=952532b0193ae630451f659ecaa3180f, ASSIGN 2023-02-08 23:54:04,111 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=952532b0193ae630451f659ecaa3180f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=false 2023-02-08 23:54:04,261 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:54:04,262 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=952532b0193ae630451f659ecaa3180f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,262 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675900444262"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900444262"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900444262"}]},"ts":"1675900444262"} 2023-02-08 23:54:04,265 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 952532b0193ae630451f659ecaa3180f, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:04,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:04,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 952532b0193ae630451f659ecaa3180f, NAME => 'testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:04,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:04,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:04,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:04,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:04,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:04,430 INFO [StoreOpener-952532b0193ae630451f659ecaa3180f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:04,432 DEBUG [StoreOpener-952532b0193ae630451f659ecaa3180f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1 2023-02-08 23:54:04,432 DEBUG [StoreOpener-952532b0193ae630451f659ecaa3180f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1 2023-02-08 23:54:04,433 INFO [StoreOpener-952532b0193ae630451f659ecaa3180f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 952532b0193ae630451f659ecaa3180f columnFamilyName cf1 2023-02-08 23:54:04,434 INFO [StoreOpener-952532b0193ae630451f659ecaa3180f-1] regionserver.HStore(310): Store=952532b0193ae630451f659ecaa3180f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:04,434 INFO [StoreOpener-952532b0193ae630451f659ecaa3180f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:04,437 DEBUG [StoreOpener-952532b0193ae630451f659ecaa3180f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2 2023-02-08 23:54:04,437 DEBUG [StoreOpener-952532b0193ae630451f659ecaa3180f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2 2023-02-08 23:54:04,438 INFO [StoreOpener-952532b0193ae630451f659ecaa3180f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 952532b0193ae630451f659ecaa3180f columnFamilyName cf2 2023-02-08 23:54:04,445 INFO [StoreOpener-952532b0193ae630451f659ecaa3180f-1] regionserver.HStore(310): Store=952532b0193ae630451f659ecaa3180f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:04,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:04,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:04,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:04,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:04,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:04,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 952532b0193ae630451f659ecaa3180f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72340908, jitterRate=0.07796353101730347}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:04,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 952532b0193ae630451f659ecaa3180f: Running coprocessor pre-open hook at 1675900444427Writing region info on filesystem at 1675900444427Initializing all the Stores at 1675900444429 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900444429Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900444430 (+1 ms)Cleaning up temporary data from old regions at 1675900444451 (+21 ms)Cleaning up detritus from prior splits at 1675900444452 (+1 ms)Running coprocessor post-open hooks at 1675900444461 (+9 ms)Region opened successfully at 1675900444461 2023-02-08 23:54:04,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f., pid=11, masterSystemTime=1675900444419 2023-02-08 23:54:04,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:04,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:04,467 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=952532b0193ae630451f659ecaa3180f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,469 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675900444467"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900444467"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900444467"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900444467"}]},"ts":"1675900444467"} 2023-02-08 23:54:04,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-08 23:54:04,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 952532b0193ae630451f659ecaa3180f, server=jenkins-hbase4.apache.org,36199,1675900441120 in 207 msec 2023-02-08 23:54:04,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-08 23:54:04,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=952532b0193ae630451f659ecaa3180f, ASSIGN in 372 msec 2023-02-08 23:54:04,484 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:54:04,484 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900444484"}]},"ts":"1675900444484"} 2023-02-08 23:54:04,487 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-08 23:54:04,492 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:54:04,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 464 msec 2023-02-08 23:54:04,562 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:04,563 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:04,566 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:04,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,574 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:04,574 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:04,575 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 23:54:04,575 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 23:54:04,576 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,576 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,577 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:04,577 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:04,577 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:04,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:04,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:04,577 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:04,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:04,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:04,578 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,578 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:04,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:04,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,579 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,579 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,579 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:04,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:04,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:04,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,579 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,579 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,580 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,581 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,581 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,581 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,582 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:04,582 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:04,582 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:04,582 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:04,582 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:04,582 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:04,583 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:04,583 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:04,583 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:04,583 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:04,584 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:04,584 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:04,584 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:04,584 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:04,584 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:04,584 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:04,585 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. started... 2023-02-08 23:54:04,586 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 952532b0193ae630451f659ecaa3180f 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 23:54:04,586 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,587 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,587 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,587 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:04,587 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,587 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:04,666 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/7cb46d4532a44eda87b1ca74d684c08e 2023-02-08 23:54:04,729 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/491d1ea3e14749ebb567f550bee27072 2023-02-08 23:54:04,738 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/7cb46d4532a44eda87b1ca74d684c08e as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/7cb46d4532a44eda87b1ca74d684c08e 2023-02-08 23:54:04,748 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/7cb46d4532a44eda87b1ca74d684c08e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:54:04,750 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/491d1ea3e14749ebb567f550bee27072 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/491d1ea3e14749ebb567f550bee27072 2023-02-08 23:54:04,759 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/491d1ea3e14749ebb567f550bee27072, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:54:04,761 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 952532b0193ae630451f659ecaa3180f in 175ms, sequenceid=6, compaction requested=false 2023-02-08 23:54:04,762 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:04,765 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 952532b0193ae630451f659ecaa3180f: 2023-02-08 23:54:04,765 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:04,765 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:04,766 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:04,766 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,766 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:04,767 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:04,774 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,774 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:04,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:04,775 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,775 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:04,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:04,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:04,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:04,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:04,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:04,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 23:54:04,778 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:04,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71016c5b[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:04,778 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,782 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,782 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,782 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,783 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:04,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,783 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:04,783 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,783 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:04,783 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:04,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,783 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:04,784 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:04,784 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:04,784 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:04,784 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:04,784 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:04,787 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:04,787 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:04,787 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:04,789 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:04,789 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:04,789 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:04,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,791 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:04,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,791 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:04,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:04,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:04,791 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:04,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:04,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:04,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:04,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:04,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:04,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:04,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:04,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:04,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:04,796 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 23:54:04,796 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:04,796 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:04,796 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,796 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:04,799 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,799 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,799 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:04,799 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,799 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,799 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:04,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,799 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:04,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:04,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:04,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:04,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:04,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:04,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:04,800 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:04,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:04,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:04,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:04,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,801 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:04,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,801 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:04,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:04,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:04,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,803 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:04,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:04,804 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:04,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,804 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:04,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:04,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,813 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,813 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,813 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:04,813 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:04,813 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:04,813 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:04,813 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:04,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:04,814 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:04,813 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:04,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:04,814 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:04,814 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:04,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:04,814 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:04,814 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:04,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:04,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:04,815 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,815 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:04,815 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:04,815 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:04,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:04,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:04,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:04,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:04,815 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:04,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:04,815 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,816 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,817 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:04,817 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:04,917 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:04,918 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:04,918 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:04,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:04,946 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:04,946 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:04,948 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:04,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:04,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,951 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:04,951 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:04,952 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 23:54:04,952 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 23:54:04,952 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,952 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,953 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,953 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:04,954 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:04,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:04,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:04,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:04,954 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:04,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:04,954 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:04,954 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:04,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:04,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:04,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:04,955 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,955 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:04,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,956 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,956 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:04,956 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:04,956 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,956 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:04,956 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:04,956 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,956 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:04,956 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:04,957 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:04,957 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:04,956 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,957 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:04,957 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:04,957 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:04,958 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:04,958 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:04,958 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:04,958 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:04,958 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:04,958 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. started... 2023-02-08 23:54:04,958 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:04,958 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:04,958 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:04,958 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 952532b0193ae630451f659ecaa3180f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:04,961 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,962 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,962 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,962 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:04,962 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:04,962 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:04,980 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/6ff65a65e21d41529a83b4d76c08aca3 2023-02-08 23:54:05,010 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/557d968cedac4aba92266d7d96b46e73 2023-02-08 23:54:05,019 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/6ff65a65e21d41529a83b4d76c08aca3 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/6ff65a65e21d41529a83b4d76c08aca3 2023-02-08 23:54:05,026 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/6ff65a65e21d41529a83b4d76c08aca3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:54:05,027 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/557d968cedac4aba92266d7d96b46e73 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/557d968cedac4aba92266d7d96b46e73 2023-02-08 23:54:05,035 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/557d968cedac4aba92266d7d96b46e73, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:54:05,037 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 952532b0193ae630451f659ecaa3180f in 79ms, sequenceid=19, compaction requested=false 2023-02-08 23:54:05,037 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 952532b0193ae630451f659ecaa3180f: 2023-02-08 23:54:05,037 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:05,037 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:05,038 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:05,038 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,038 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:05,038 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:05,040 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,040 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:05,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:05,040 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,041 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:05,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:05,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:05,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:05,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 23:54:05,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a9df297[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:05,043 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:05,043 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,045 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,045 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,045 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,045 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:05,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,045 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,045 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:05,045 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:05,045 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:05,045 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:05,045 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:05,045 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:05,045 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:05,045 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:05,046 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:05,049 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,049 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:05,049 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,049 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:05,049 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:05,049 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,050 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:05,050 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:05,050 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:05,050 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:05,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:05,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:05,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:05,055 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:05,055 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 23:54:05,055 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:05,055 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:05,055 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,055 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:05,057 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,057 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,057 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:05,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:05,057 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,057 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,058 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,058 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,058 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:05,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,059 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,060 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:05,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,063 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:05,063 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,064 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,074 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,074 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,074 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:05,074 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:05,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,074 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,074 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,075 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:05,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,075 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:05,075 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,075 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,075 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,075 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,075 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,075 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,075 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,076 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,175 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:05,175 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:05,175 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:05,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:05,199 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:05,199 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:05,200 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:05,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:05,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,203 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:05,203 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,203 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 23:54:05,203 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 23:54:05,204 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,204 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,205 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,205 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,205 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,205 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,206 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,206 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:05,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,206 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,206 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:05,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:05,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:05,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,207 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,207 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,207 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,207 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:05,207 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,207 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:05,207 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:05,207 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:05,208 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,208 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,208 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:05,208 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:05,208 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,208 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:05,209 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:05,209 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:05,208 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,209 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,209 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:05,209 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:05,210 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:05,209 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:05,210 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:05,209 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:05,210 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. started... 2023-02-08 23:54:05,210 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:05,210 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 952532b0193ae630451f659ecaa3180f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:05,212 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,213 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,213 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:05,213 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,214 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,214 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:05,229 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/1e01050191d74f09b5dbc6d0c0873826 2023-02-08 23:54:05,252 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/ba29610f803a44398a6f0e1eec52390c 2023-02-08 23:54:05,260 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/1e01050191d74f09b5dbc6d0c0873826 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/1e01050191d74f09b5dbc6d0c0873826 2023-02-08 23:54:05,267 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/1e01050191d74f09b5dbc6d0c0873826, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:54:05,269 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/ba29610f803a44398a6f0e1eec52390c as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ba29610f803a44398a6f0e1eec52390c 2023-02-08 23:54:05,276 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ba29610f803a44398a6f0e1eec52390c, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:54:05,277 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 952532b0193ae630451f659ecaa3180f in 67ms, sequenceid=32, compaction requested=true 2023-02-08 23:54:05,277 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 952532b0193ae630451f659ecaa3180f: 2023-02-08 23:54:05,277 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:05,280 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,280 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:05,280 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,280 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:05,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 23:54:05,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@493c792c[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:05,283 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:05,283 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,284 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,284 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,284 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,285 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,285 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:05,290 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:05,290 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,290 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:05,290 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:05,290 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,290 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:05,291 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:05,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:05,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:05,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:05,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:05,295 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 23:54:05,295 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:05,295 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:05,295 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,295 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:05,297 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,297 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,297 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,297 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,297 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,297 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:05,297 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,297 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:05,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,298 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,298 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:05,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,301 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:05,301 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,301 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,305 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,305 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,305 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,305 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,305 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,305 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,305 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,305 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,305 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,306 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,306 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,306 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:05,306 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:05,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,306 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,307 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,307 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,307 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:05,307 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,307 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:05,307 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,308 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,308 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,408 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:05,408 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:05,408 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:05,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:05,430 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:05,431 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:05,431 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:05,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:05,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,434 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:05,434 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,435 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 23:54:05,435 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 23:54:05,436 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,436 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,437 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,437 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,437 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,437 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,438 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,438 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:05,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,439 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,439 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:05,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:05,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:05,439 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,439 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,439 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,439 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:05,439 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,439 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:05,439 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:05,440 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,439 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,440 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:05,441 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:05,441 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:05,441 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,441 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:05,441 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:05,442 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:05,442 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:05,442 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:05,442 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:05,442 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:05,443 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. started... 2023-02-08 23:54:05,443 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 952532b0193ae630451f659ecaa3180f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:05,445 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:05,461 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/2155b6bd47974950b0ce4afc663c4f3f 2023-02-08 23:54:05,486 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/c9c0dec0d6b94ddf901872809a0343fa 2023-02-08 23:54:05,494 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/2155b6bd47974950b0ce4afc663c4f3f as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/2155b6bd47974950b0ce4afc663c4f3f 2023-02-08 23:54:05,501 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/2155b6bd47974950b0ce4afc663c4f3f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:54:05,502 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/c9c0dec0d6b94ddf901872809a0343fa as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/c9c0dec0d6b94ddf901872809a0343fa 2023-02-08 23:54:05,509 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/c9c0dec0d6b94ddf901872809a0343fa, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:54:05,510 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 952532b0193ae630451f659ecaa3180f in 67ms, sequenceid=45, compaction requested=true 2023-02-08 23:54:05,510 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 952532b0193ae630451f659ecaa3180f: 2023-02-08 23:54:05,510 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:05,510 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:05,511 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:05,511 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,511 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:05,511 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:05,517 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,517 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:05,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:05,517 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,517 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:05,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:05,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:05,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:05,519 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 23:54:05,519 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b43a4fe[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:05,519 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:05,519 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,520 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,520 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,520 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,521 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,521 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:05,521 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:05,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,521 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:05,521 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:05,521 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:05,521 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:05,521 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:05,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,521 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:05,521 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:05,521 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:05,522 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:05,522 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,522 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:05,524 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:05,524 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,524 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:05,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:05,524 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:05,525 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:05,525 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,525 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:05,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:05,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:05,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:05,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:05,529 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:05,529 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 23:54:05,529 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:05,529 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:05,529 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,529 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:05,530 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,530 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,530 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,530 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,530 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,530 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:05,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:05,531 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,530 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:05,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,531 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,532 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:05,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,534 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:05,534 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,534 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,537 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,537 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,537 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,537 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,538 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,537 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,538 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:05,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,538 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,537 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,538 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,538 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,538 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:05,538 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,539 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,539 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,539 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,539 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:05,539 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,540 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:05,540 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,540 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,540 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,640 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:05,640 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:05,640 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:05,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:05,663 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:05,663 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:05,664 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:05,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:05,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,666 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:05,666 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,667 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 23:54:05,667 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 23:54:05,667 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,668 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,670 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,670 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,670 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:05,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,671 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,671 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,671 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:05,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:05,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:05,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,672 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,672 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,672 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,672 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:05,672 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,672 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:05,672 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:05,672 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,672 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:05,674 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:05,674 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,674 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,675 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:05,675 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:05,674 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,675 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:05,675 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:05,675 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:05,675 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,675 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:05,677 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:05,677 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:05,677 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:05,677 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:05,677 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. started... 2023-02-08 23:54:05,677 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:05,677 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 952532b0193ae630451f659ecaa3180f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:05,678 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,679 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,680 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,680 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:05,680 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,680 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:05,697 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/8fc6b0350c1744baac8be7dd29977be0 2023-02-08 23:54:05,746 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/edb7db88f93649d7b79caecab104fc6d 2023-02-08 23:54:05,755 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/8fc6b0350c1744baac8be7dd29977be0 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/8fc6b0350c1744baac8be7dd29977be0 2023-02-08 23:54:05,762 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/8fc6b0350c1744baac8be7dd29977be0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:54:05,763 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/edb7db88f93649d7b79caecab104fc6d as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/edb7db88f93649d7b79caecab104fc6d 2023-02-08 23:54:05,770 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/edb7db88f93649d7b79caecab104fc6d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:54:05,771 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 952532b0193ae630451f659ecaa3180f in 94ms, sequenceid=58, compaction requested=true 2023-02-08 23:54:05,771 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 952532b0193ae630451f659ecaa3180f: 2023-02-08 23:54:05,771 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:05,771 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:05,772 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:05,772 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,772 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:05,772 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:05,775 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,775 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:05,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:05,775 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,775 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:05,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:05,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:05,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:05,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 23:54:05,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61da7079[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:05,778 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:05,778 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,779 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,779 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,779 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,779 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:05,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,779 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:05,779 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:05,779 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:05,779 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:05,779 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,779 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:05,779 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:05,779 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:05,779 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:05,780 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:05,781 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:05,781 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,781 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:05,782 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:05,782 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,782 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:05,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:05,782 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:05,782 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:05,783 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,783 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:05,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:05,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:05,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:05,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:05,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:05,788 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 23:54:05,788 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:05,788 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:05,788 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,788 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:05,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,792 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,792 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:05,793 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,794 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,795 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:05,795 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,806 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,806 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,806 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,806 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,806 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,806 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,806 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,806 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:05,806 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:05,806 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,806 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,806 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,806 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,807 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:05,807 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:05,807 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:05,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:05,807 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,807 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,807 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:05,807 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:05,808 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,808 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,907 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:05,908 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:05,908 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:05,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:05,929 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:05,929 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:05,930 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:05,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:05,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,932 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:05,932 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,933 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 23:54:05,933 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 23:54:05,934 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,934 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,935 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,935 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,935 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:05,935 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:05,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:05,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,936 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:05,936 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:05,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,936 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,936 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,936 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:05,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:05,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:05,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,937 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:05,937 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,937 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:05,937 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:05,937 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 23:54:05,937 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,937 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,938 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:05,938 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:05,938 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:05,938 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:05,937 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,938 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,938 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 23:54:05,939 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:05,939 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:05,939 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 23:54:05,939 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:05,939 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:05,939 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:05,939 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:05,939 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:05,939 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. started... 2023-02-08 23:54:05,940 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 952532b0193ae630451f659ecaa3180f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:05,941 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,942 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,942 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,942 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:05,942 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:05,942 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:05,958 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/20c8043f893b45838a512a3d97be16ba 2023-02-08 23:54:05,977 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/0f9fe5fb95b047d5b0bd7a7d920c7169 2023-02-08 23:54:05,984 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/20c8043f893b45838a512a3d97be16ba as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/20c8043f893b45838a512a3d97be16ba 2023-02-08 23:54:05,991 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/20c8043f893b45838a512a3d97be16ba, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:54:05,992 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/0f9fe5fb95b047d5b0bd7a7d920c7169 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/0f9fe5fb95b047d5b0bd7a7d920c7169 2023-02-08 23:54:05,998 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/0f9fe5fb95b047d5b0bd7a7d920c7169, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:54:05,999 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 952532b0193ae630451f659ecaa3180f in 59ms, sequenceid=71, compaction requested=true 2023-02-08 23:54:05,999 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 952532b0193ae630451f659ecaa3180f: 2023-02-08 23:54:06,000 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:06,000 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:06,001 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:06,001 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,001 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 23:54:06,001 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 23:54:06,003 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,003 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:06,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:06,003 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,003 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:06,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:06,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:06,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:06,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:06,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:06,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 23:54:06,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24162f9f[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:06,005 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:06,005 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,006 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,006 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,006 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,007 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:06,007 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:06,007 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:06,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,007 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,007 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:06,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,007 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:06,007 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 23:54:06,007 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:06,007 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:06,007 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 23:54:06,007 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:06,008 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,008 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:06,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:06,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:06,008 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:06,009 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:06,009 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:06,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:06,010 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 23:54:06,010 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:06,010 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:06,010 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:06,010 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 23:54:06,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:06,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:06,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:06,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:06,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:06,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:06,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:06,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:06,014 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 23:54:06,014 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:06,014 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:06,014 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,014 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:06,017 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,017 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,017 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:06,017 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:06,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,017 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,017 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,017 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,017 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:06,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:06,018 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:06,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,018 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:06,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:06,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:06,020 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:06,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,020 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:06,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:06,021 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:06,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:06,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,024 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,024 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,024 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:06,024 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:06,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:06,024 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:06,024 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:06,024 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:06,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:06,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:06,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:06,024 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 23:54:06,025 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:06,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:06,024 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:06,024 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:06,024 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:06,025 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:06,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:06,026 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:06,024 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:06,026 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:06,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:06,026 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:06,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:06,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:06,026 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:06,026 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:06,026 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:06,026 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,027 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 23:54:06,126 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:06,126 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:06,126 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:06,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:06,149 DEBUG [Listener at localhost/37973] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,155 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:06,156 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:06,156 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:06,156 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:06,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=952532b0193ae630451f659ecaa3180f, UNASSIGN}] 2023-02-08 23:54:06,165 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=952532b0193ae630451f659ecaa3180f, UNASSIGN 2023-02-08 23:54:06,165 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=952532b0193ae630451f659ecaa3180f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:06,166 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675900446165"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900446165"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900446165"}]},"ts":"1675900446165"} 2023-02-08 23:54:06,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 952532b0193ae630451f659ecaa3180f, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:06,328 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 952532b0193ae630451f659ecaa3180f, disabling compactions & flushes 2023-02-08 23:54:06,328 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:06,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:06,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. after waiting 0 ms 2023-02-08 23:54:06,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:06,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 952532b0193ae630451f659ecaa3180f 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 23:54:06,346 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/b635daea7a254c7cbb9341b3b5537aa4 2023-02-08 23:54:06,368 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/ee6b8b3d7084428fb4201ff37c7606a2 2023-02-08 23:54:06,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf1/b635daea7a254c7cbb9341b3b5537aa4 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/b635daea7a254c7cbb9341b3b5537aa4 2023-02-08 23:54:06,386 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/b635daea7a254c7cbb9341b3b5537aa4, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:54:06,388 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.tmp/cf2/ee6b8b3d7084428fb4201ff37c7606a2 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ee6b8b3d7084428fb4201ff37c7606a2 2023-02-08 23:54:06,397 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ee6b8b3d7084428fb4201ff37c7606a2, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:54:06,398 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 952532b0193ae630451f659ecaa3180f in 69ms, sequenceid=82, compaction requested=true 2023-02-08 23:54:06,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 23:54:06,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:06,420 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. 2023-02-08 23:54:06,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 952532b0193ae630451f659ecaa3180f: Waiting for close lock at 1675900446328Running coprocessor pre-close hooks at 1675900446328Disabling compacts and flushes for region at 1675900446328Disabling writes for close at 1675900446328Obtaining lock to block concurrent updates at 1675900446329 (+1 ms)Preparing flush snapshotting stores in 952532b0193ae630451f659ecaa3180f at 1675900446329Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675900446329Flushing stores of testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. at 1675900446330 (+1 ms)Flushing 952532b0193ae630451f659ecaa3180f/cf1: creating writer at 1675900446330Flushing 952532b0193ae630451f659ecaa3180f/cf1: appending metadata at 1675900446333 (+3 ms)Flushing 952532b0193ae630451f659ecaa3180f/cf1: closing flushed file at 1675900446333Flushing 952532b0193ae630451f659ecaa3180f/cf2: creating writer at 1675900446353 (+20 ms)Flushing 952532b0193ae630451f659ecaa3180f/cf2: appending metadata at 1675900446356 (+3 ms)Flushing 952532b0193ae630451f659ecaa3180f/cf2: closing flushed file at 1675900446356Flushing 952532b0193ae630451f659ecaa3180f/cf1: reopening flushed file at 1675900446380 (+24 ms)Flushing 952532b0193ae630451f659ecaa3180f/cf2: reopening flushed file at 1675900446390 (+10 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 952532b0193ae630451f659ecaa3180f in 69ms, sequenceid=82, compaction requested=true at 1675900446398 (+8 ms)Writing region close event to WAL at 1675900446414 (+16 ms)Running coprocessor post-close hooks at 1675900446420 (+6 ms)Closed at 1675900446420 2023-02-08 23:54:06,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,424 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=952532b0193ae630451f659ecaa3180f, regionState=CLOSED 2023-02-08 23:54:06,425 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675900446424"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900446424"}]},"ts":"1675900446424"} 2023-02-08 23:54:06,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-08 23:54:06,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 952532b0193ae630451f659ecaa3180f, server=jenkins-hbase4.apache.org,36199,1675900441120 in 259 msec 2023-02-08 23:54:06,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-08 23:54:06,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=952532b0193ae630451f659ecaa3180f, UNASSIGN in 267 msec 2023-02-08 23:54:06,449 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=952532b0193ae630451f659ecaa3180f, threads=14 2023-02-08 23:54:06,450 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/0f9fe5fb95b047d5b0bd7a7d920c7169 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,450 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/491d1ea3e14749ebb567f550bee27072 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,452 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/557d968cedac4aba92266d7d96b46e73 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,452 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ba29610f803a44398a6f0e1eec52390c for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,452 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/c9c0dec0d6b94ddf901872809a0343fa for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,453 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/edb7db88f93649d7b79caecab104fc6d for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,455 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ee6b8b3d7084428fb4201ff37c7606a2 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,455 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/1e01050191d74f09b5dbc6d0c0873826 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,455 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/20c8043f893b45838a512a3d97be16ba for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,455 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/2155b6bd47974950b0ce4afc663c4f3f for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,456 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/6ff65a65e21d41529a83b4d76c08aca3 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,456 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/7cb46d4532a44eda87b1ca74d684c08e for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,456 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/8fc6b0350c1744baac8be7dd29977be0 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,457 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/b635daea7a254c7cbb9341b3b5537aa4 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,472 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/0f9fe5fb95b047d5b0bd7a7d920c7169, top=true 2023-02-08 23:54:06,513 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/8fc6b0350c1744baac8be7dd29977be0, top=true 2023-02-08 23:54:06,514 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/edb7db88f93649d7b79caecab104fc6d, top=true 2023-02-08 23:54:06,542 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/b635daea7a254c7cbb9341b3b5537aa4, top=true 2023-02-08 23:54:06,546 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.splits/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-0f9fe5fb95b047d5b0bd7a7d920c7169 for child: 35b9c4ae42935ae148dc156df868a3bc, parent: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,547 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/0f9fe5fb95b047d5b0bd7a7d920c7169 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,547 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/20c8043f893b45838a512a3d97be16ba, top=true 2023-02-08 23:54:06,555 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ee6b8b3d7084428fb4201ff37c7606a2, top=true 2023-02-08 23:54:06,575 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.splits/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-8fc6b0350c1744baac8be7dd29977be0 for child: 35b9c4ae42935ae148dc156df868a3bc, parent: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,575 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/8fc6b0350c1744baac8be7dd29977be0 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,582 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.splits/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-edb7db88f93649d7b79caecab104fc6d for child: 35b9c4ae42935ae148dc156df868a3bc, parent: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,582 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/edb7db88f93649d7b79caecab104fc6d for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,592 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.splits/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-b635daea7a254c7cbb9341b3b5537aa4 for child: 35b9c4ae42935ae148dc156df868a3bc, parent: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,592 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/b635daea7a254c7cbb9341b3b5537aa4 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,593 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.splits/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-20c8043f893b45838a512a3d97be16ba for child: 35b9c4ae42935ae148dc156df868a3bc, parent: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,593 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/20c8043f893b45838a512a3d97be16ba for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,598 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/.splits/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-ee6b8b3d7084428fb4201ff37c7606a2 for child: 35b9c4ae42935ae148dc156df868a3bc, parent: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,598 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ee6b8b3d7084428fb4201ff37c7606a2 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,628 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/c9c0dec0d6b94ddf901872809a0343fa for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,647 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/557d968cedac4aba92266d7d96b46e73 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,651 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/6ff65a65e21d41529a83b4d76c08aca3 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,653 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/1e01050191d74f09b5dbc6d0c0873826 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:06,658 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/7cb46d4532a44eda87b1ca74d684c08e for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:07,018 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ba29610f803a44398a6f0e1eec52390c for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:07,050 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/491d1ea3e14749ebb567f550bee27072 for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:07,417 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/2155b6bd47974950b0ce4afc663c4f3f for region: 952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:07,417 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 952532b0193ae630451f659ecaa3180f Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-08 23:54:07,469 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 23:54:07,471 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 23:54:07,474 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675900447473"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1675900447473"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1675900447473"}]},"ts":"1675900447473"} 2023-02-08 23:54:07,474 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900447473"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900447473"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900447473"}]},"ts":"1675900447473"} 2023-02-08 23:54:07,474 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900447473"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900447473"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900447473"}]},"ts":"1675900447473"} 2023-02-08 23:54:07,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5981761d0d9fef4c60f7e5990070c751, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=35b9c4ae42935ae148dc156df868a3bc, ASSIGN}] 2023-02-08 23:54:07,523 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=35b9c4ae42935ae148dc156df868a3bc, ASSIGN 2023-02-08 23:54:07,524 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=35b9c4ae42935ae148dc156df868a3bc, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=false 2023-02-08 23:54:07,525 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5981761d0d9fef4c60f7e5990070c751, ASSIGN 2023-02-08 23:54:07,526 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5981761d0d9fef4c60f7e5990070c751, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=false 2023-02-08 23:54:07,675 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 23:54:07,676 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=5981761d0d9fef4c60f7e5990070c751, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:07,677 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900447676"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900447676"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900447676"}]},"ts":"1675900447676"} 2023-02-08 23:54:07,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 5981761d0d9fef4c60f7e5990070c751, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:07,684 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=35b9c4ae42935ae148dc156df868a3bc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:07,684 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900447684"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900447684"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900447684"}]},"ts":"1675900447684"} 2023-02-08 23:54:07,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 35b9c4ae42935ae148dc156df868a3bc, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:07,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751. 2023-02-08 23:54:07,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5981761d0d9fef4c60f7e5990070c751, NAME => 'testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.', STARTKEY => '', ENDKEY => '26'} 2023-02-08 23:54:07,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:07,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 5981761d0d9fef4c60f7e5990070c751 2023-02-08 23:54:07,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:07,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 5981761d0d9fef4c60f7e5990070c751 2023-02-08 23:54:07,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 5981761d0d9fef4c60f7e5990070c751 2023-02-08 23:54:07,841 INFO [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5981761d0d9fef4c60f7e5990070c751 2023-02-08 23:54:07,842 DEBUG [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1 2023-02-08 23:54:07,842 DEBUG [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1 2023-02-08 23:54:07,843 INFO [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5981761d0d9fef4c60f7e5990070c751 columnFamilyName cf1 2023-02-08 23:54:07,881 DEBUG [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/1e01050191d74f09b5dbc6d0c0873826.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/1e01050191d74f09b5dbc6d0c0873826-bottom 2023-02-08 23:54:07,910 DEBUG [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/2155b6bd47974950b0ce4afc663c4f3f.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/2155b6bd47974950b0ce4afc663c4f3f-bottom 2023-02-08 23:54:07,918 DEBUG [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/6ff65a65e21d41529a83b4d76c08aca3.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/6ff65a65e21d41529a83b4d76c08aca3-bottom 2023-02-08 23:54:07,924 DEBUG [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/7cb46d4532a44eda87b1ca74d684c08e.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/7cb46d4532a44eda87b1ca74d684c08e-bottom 2023-02-08 23:54:07,924 INFO [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] regionserver.HStore(310): Store=5981761d0d9fef4c60f7e5990070c751/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:07,925 INFO [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5981761d0d9fef4c60f7e5990070c751 2023-02-08 23:54:07,926 DEBUG [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2 2023-02-08 23:54:07,927 DEBUG [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2 2023-02-08 23:54:07,927 INFO [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5981761d0d9fef4c60f7e5990070c751 columnFamilyName cf2 2023-02-08 23:54:07,942 DEBUG [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/491d1ea3e14749ebb567f550bee27072.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/491d1ea3e14749ebb567f550bee27072-bottom 2023-02-08 23:54:07,948 DEBUG [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/557d968cedac4aba92266d7d96b46e73.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/557d968cedac4aba92266d7d96b46e73-bottom 2023-02-08 23:54:07,954 DEBUG [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/ba29610f803a44398a6f0e1eec52390c.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ba29610f803a44398a6f0e1eec52390c-bottom 2023-02-08 23:54:07,960 DEBUG [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/c9c0dec0d6b94ddf901872809a0343fa.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/c9c0dec0d6b94ddf901872809a0343fa-bottom 2023-02-08 23:54:07,960 INFO [StoreOpener-5981761d0d9fef4c60f7e5990070c751-1] regionserver.HStore(310): Store=5981761d0d9fef4c60f7e5990070c751/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:07,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751 2023-02-08 23:54:07,963 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751 2023-02-08 23:54:07,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:07,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 5981761d0d9fef4c60f7e5990070c751 2023-02-08 23:54:07,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 5981761d0d9fef4c60f7e5990070c751; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70771247, jitterRate=0.05457375943660736}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:07,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 5981761d0d9fef4c60f7e5990070c751: Running coprocessor pre-open hook at 1675900447839Writing region info on filesystem at 1675900447839Initializing all the Stores at 1675900447840 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900447841 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900447841Cleaning up temporary data from old regions at 1675900447964 (+123 ms)Cleaning up detritus from prior splits at 1675900447964Running coprocessor post-open hooks at 1675900447968 (+4 ms)Region opened successfully at 1675900447968 2023-02-08 23:54:07,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751., pid=17, masterSystemTime=1675900447832 2023-02-08 23:54:07,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:07,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:54:07,972 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 23:54:07,972 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 23:54:07,975 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751. 2023-02-08 23:54:07,975 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751. 2023-02-08 23:54:07,975 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1912): 5981761d0d9fef4c60f7e5990070c751/cf1 is initiating minor compaction (all files) 2023-02-08 23:54:07,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751. 2023-02-08 23:54:07,975 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5981761d0d9fef4c60f7e5990070c751/cf1 in testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751. 2023-02-08 23:54:07,975 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1912): 5981761d0d9fef4c60f7e5990070c751/cf2 is initiating minor compaction (all files) 2023-02-08 23:54:07,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751. 2023-02-08 23:54:07,976 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5981761d0d9fef4c60f7e5990070c751/cf2 in testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751. 2023-02-08 23:54:07,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc. 2023-02-08 23:54:07,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 35b9c4ae42935ae148dc156df868a3bc, NAME => 'testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.', STARTKEY => '26', ENDKEY => ''} 2023-02-08 23:54:07,976 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=5981761d0d9fef4c60f7e5990070c751, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:07,976 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/7cb46d4532a44eda87b1ca74d684c08e.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/7cb46d4532a44eda87b1ca74d684c08e-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/6ff65a65e21d41529a83b4d76c08aca3.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/6ff65a65e21d41529a83b4d76c08aca3-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/1e01050191d74f09b5dbc6d0c0873826.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/1e01050191d74f09b5dbc6d0c0873826-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/2155b6bd47974950b0ce4afc663c4f3f.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/2155b6bd47974950b0ce4afc663c4f3f-bottom] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/.tmp, totalSize=19.9 K 2023-02-08 23:54:07,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:07,976 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900447976"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900447976"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900447976"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900447976"}]},"ts":"1675900447976"} 2023-02-08 23:54:07,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:07,976 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/491d1ea3e14749ebb567f550bee27072.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/491d1ea3e14749ebb567f550bee27072-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/557d968cedac4aba92266d7d96b46e73.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/557d968cedac4aba92266d7d96b46e73-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/ba29610f803a44398a6f0e1eec52390c.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ba29610f803a44398a6f0e1eec52390c-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/c9c0dec0d6b94ddf901872809a0343fa.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/c9c0dec0d6b94ddf901872809a0343fa-bottom] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/.tmp, totalSize=19.9 K 2023-02-08 23:54:07,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:07,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:07,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:07,979 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 7cb46d4532a44eda87b1ca74d684c08e.952532b0193ae630451f659ecaa3180f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900444538 2023-02-08 23:54:07,979 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting 491d1ea3e14749ebb567f550bee27072.952532b0193ae630451f659ecaa3180f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900444538 2023-02-08 23:54:07,980 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 6ff65a65e21d41529a83b4d76c08aca3.952532b0193ae630451f659ecaa3180f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900444925 2023-02-08 23:54:07,981 INFO [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:07,981 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting 557d968cedac4aba92266d7d96b46e73.952532b0193ae630451f659ecaa3180f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900444925 2023-02-08 23:54:07,981 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting ba29610f803a44398a6f0e1eec52390c.952532b0193ae630451f659ecaa3180f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900445178 2023-02-08 23:54:07,981 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 1e01050191d74f09b5dbc6d0c0873826.952532b0193ae630451f659ecaa3180f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900445178 2023-02-08 23:54:07,982 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1 2023-02-08 23:54:07,982 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1 2023-02-08 23:54:07,983 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting c9c0dec0d6b94ddf901872809a0343fa.952532b0193ae630451f659ecaa3180f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900445410 2023-02-08 23:54:07,983 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 2155b6bd47974950b0ce4afc663c4f3f.952532b0193ae630451f659ecaa3180f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900445410 2023-02-08 23:54:07,983 INFO [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35b9c4ae42935ae148dc156df868a3bc columnFamilyName cf1 2023-02-08 23:54:07,985 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-08 23:54:07,985 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 5981761d0d9fef4c60f7e5990070c751, server=jenkins-hbase4.apache.org,36199,1675900441120 in 302 msec 2023-02-08 23:54:07,987 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5981761d0d9fef4c60f7e5990070c751, ASSIGN in 463 msec 2023-02-08 23:54:08,003 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/1e01050191d74f09b5dbc6d0c0873826.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/1e01050191d74f09b5dbc6d0c0873826-top 2023-02-08 23:54:08,004 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5981761d0d9fef4c60f7e5990070c751#cf1#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:08,010 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] throttle.PressureAwareThroughputController(145): 5981761d0d9fef4c60f7e5990070c751#cf2#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:08,011 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/2155b6bd47974950b0ce4afc663c4f3f.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/2155b6bd47974950b0ce4afc663c4f3f-top 2023-02-08 23:54:08,017 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/6ff65a65e21d41529a83b4d76c08aca3.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/6ff65a65e21d41529a83b4d76c08aca3-top 2023-02-08 23:54:08,022 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/7cb46d4532a44eda87b1ca74d684c08e.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/7cb46d4532a44eda87b1ca74d684c08e-top 2023-02-08 23:54:08,028 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-20c8043f893b45838a512a3d97be16ba 2023-02-08 23:54:08,037 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-8fc6b0350c1744baac8be7dd29977be0 2023-02-08 23:54:08,050 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-b635daea7a254c7cbb9341b3b5537aa4 2023-02-08 23:54:08,050 INFO [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(310): Store=35b9c4ae42935ae148dc156df868a3bc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:08,051 INFO [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:08,052 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/.tmp/cf2/7c56875488d04069851e8143178bbb25 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/7c56875488d04069851e8143178bbb25 2023-02-08 23:54:08,053 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2 2023-02-08 23:54:08,053 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2 2023-02-08 23:54:08,056 INFO [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35b9c4ae42935ae148dc156df868a3bc columnFamilyName cf2 2023-02-08 23:54:08,057 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/.tmp/cf1/311b9e2d0af24caab497c8c4c1cefe56 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/311b9e2d0af24caab497c8c4c1cefe56 2023-02-08 23:54:08,073 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5981761d0d9fef4c60f7e5990070c751/cf2 of 5981761d0d9fef4c60f7e5990070c751 into 7c56875488d04069851e8143178bbb25(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:08,073 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5981761d0d9fef4c60f7e5990070c751/cf1 of 5981761d0d9fef4c60f7e5990070c751 into 311b9e2d0af24caab497c8c4c1cefe56(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:08,075 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5981761d0d9fef4c60f7e5990070c751: 2023-02-08 23:54:08,075 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5981761d0d9fef4c60f7e5990070c751: 2023-02-08 23:54:08,075 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751., storeName=5981761d0d9fef4c60f7e5990070c751/cf2, priority=12, startTime=1675900447972; duration=0sec 2023-02-08 23:54:08,075 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751., storeName=5981761d0d9fef4c60f7e5990070c751/cf1, priority=12, startTime=1675900447971; duration=0sec 2023-02-08 23:54:08,076 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:08,076 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:08,077 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/491d1ea3e14749ebb567f550bee27072.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/491d1ea3e14749ebb567f550bee27072-top 2023-02-08 23:54:08,083 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/557d968cedac4aba92266d7d96b46e73.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/557d968cedac4aba92266d7d96b46e73-top 2023-02-08 23:54:08,088 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/ba29610f803a44398a6f0e1eec52390c.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ba29610f803a44398a6f0e1eec52390c-top 2023-02-08 23:54:08,094 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/c9c0dec0d6b94ddf901872809a0343fa.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/c9c0dec0d6b94ddf901872809a0343fa-top 2023-02-08 23:54:08,098 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-0f9fe5fb95b047d5b0bd7a7d920c7169 2023-02-08 23:54:08,103 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-edb7db88f93649d7b79caecab104fc6d 2023-02-08 23:54:08,108 DEBUG [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-ee6b8b3d7084428fb4201ff37c7606a2 2023-02-08 23:54:08,108 INFO [StoreOpener-35b9c4ae42935ae148dc156df868a3bc-1] regionserver.HStore(310): Store=35b9c4ae42935ae148dc156df868a3bc/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:08,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:08,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:08,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:08,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:08,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 35b9c4ae42935ae148dc156df868a3bc; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59576455, jitterRate=-0.11224164068698883}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:08,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 35b9c4ae42935ae148dc156df868a3bc: Running coprocessor pre-open hook at 1675900447978Writing region info on filesystem at 1675900447978Initializing all the Stores at 1675900447980 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900447980Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900447980Cleaning up temporary data from old regions at 1675900448112 (+132 ms)Cleaning up detritus from prior splits at 1675900448113 (+1 ms)Running coprocessor post-open hooks at 1675900448116 (+3 ms)Region opened successfully at 1675900448116 2023-02-08 23:54:08,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc., pid=18, masterSystemTime=1675900447832 2023-02-08 23:54:08,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:08,118 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:54:08,121 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:54:08,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:08,121 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc. 2023-02-08 23:54:08,121 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc. 2023-02-08 23:54:08,121 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1912): 35b9c4ae42935ae148dc156df868a3bc/cf2 is initiating minor compaction (all files) 2023-02-08 23:54:08,121 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1912): 35b9c4ae42935ae148dc156df868a3bc/cf1 is initiating minor compaction (all files) 2023-02-08 23:54:08,121 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 35b9c4ae42935ae148dc156df868a3bc/cf2 in testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc. 2023-02-08 23:54:08,121 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 35b9c4ae42935ae148dc156df868a3bc/cf1 in testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc. 2023-02-08 23:54:08,122 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/491d1ea3e14749ebb567f550bee27072.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/491d1ea3e14749ebb567f550bee27072-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/557d968cedac4aba92266d7d96b46e73.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/557d968cedac4aba92266d7d96b46e73-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/ba29610f803a44398a6f0e1eec52390c.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ba29610f803a44398a6f0e1eec52390c-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/c9c0dec0d6b94ddf901872809a0343fa.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/c9c0dec0d6b94ddf901872809a0343fa-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-edb7db88f93649d7b79caecab104fc6d, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-0f9fe5fb95b047d5b0bd7a7d920c7169, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-ee6b8b3d7084428fb4201ff37c7606a2] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/.tmp, totalSize=34.9 K 2023-02-08 23:54:08,122 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/7cb46d4532a44eda87b1ca74d684c08e.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/7cb46d4532a44eda87b1ca74d684c08e-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/6ff65a65e21d41529a83b4d76c08aca3.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/6ff65a65e21d41529a83b4d76c08aca3-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/1e01050191d74f09b5dbc6d0c0873826.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/1e01050191d74f09b5dbc6d0c0873826-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/2155b6bd47974950b0ce4afc663c4f3f.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/2155b6bd47974950b0ce4afc663c4f3f-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-8fc6b0350c1744baac8be7dd29977be0, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-20c8043f893b45838a512a3d97be16ba, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-b635daea7a254c7cbb9341b3b5537aa4] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/.tmp, totalSize=34.9 K 2023-02-08 23:54:08,123 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting 491d1ea3e14749ebb567f550bee27072.952532b0193ae630451f659ecaa3180f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675900444538 2023-02-08 23:54:08,123 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 7cb46d4532a44eda87b1ca74d684c08e.952532b0193ae630451f659ecaa3180f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675900444538 2023-02-08 23:54:08,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc. 2023-02-08 23:54:08,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc. 2023-02-08 23:54:08,124 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting 557d968cedac4aba92266d7d96b46e73.952532b0193ae630451f659ecaa3180f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675900444925 2023-02-08 23:54:08,124 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 6ff65a65e21d41529a83b4d76c08aca3.952532b0193ae630451f659ecaa3180f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675900444925 2023-02-08 23:54:08,125 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=35b9c4ae42935ae148dc156df868a3bc, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:08,125 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting ba29610f803a44398a6f0e1eec52390c.952532b0193ae630451f659ecaa3180f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675900445178 2023-02-08 23:54:08,125 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 1e01050191d74f09b5dbc6d0c0873826.952532b0193ae630451f659ecaa3180f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675900445178 2023-02-08 23:54:08,125 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900448125"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900448125"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900448125"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900448125"}]},"ts":"1675900448125"} 2023-02-08 23:54:08,126 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting c9c0dec0d6b94ddf901872809a0343fa.952532b0193ae630451f659ecaa3180f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675900445410 2023-02-08 23:54:08,126 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 2155b6bd47974950b0ce4afc663c4f3f.952532b0193ae630451f659ecaa3180f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675900445410 2023-02-08 23:54:08,126 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-edb7db88f93649d7b79caecab104fc6d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900445643 2023-02-08 23:54:08,126 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-8fc6b0350c1744baac8be7dd29977be0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900445643 2023-02-08 23:54:08,127 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-0f9fe5fb95b047d5b0bd7a7d920c7169, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900445910 2023-02-08 23:54:08,127 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-20c8043f893b45838a512a3d97be16ba, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900445910 2023-02-08 23:54:08,128 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-ee6b8b3d7084428fb4201ff37c7606a2, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675900446129 2023-02-08 23:54:08,129 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-b635daea7a254c7cbb9341b3b5537aa4, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675900446129 2023-02-08 23:54:08,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-02-08 23:54:08,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 35b9c4ae42935ae148dc156df868a3bc, server=jenkins-hbase4.apache.org,36199,1675900441120 in 441 msec 2023-02-08 23:54:08,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-02-08 23:54:08,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=35b9c4ae42935ae148dc156df868a3bc, ASSIGN in 610 msec 2023-02-08 23:54:08,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc in 1.9860 sec 2023-02-08 23:54:08,154 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] throttle.PressureAwareThroughputController(145): 35b9c4ae42935ae148dc156df868a3bc#cf2#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:08,154 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] throttle.PressureAwareThroughputController(145): 35b9c4ae42935ae148dc156df868a3bc#cf1#compaction#17 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:08,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36199] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:53688 deadline: 1675900508161, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f. is not online on jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:08,179 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/.tmp/cf2/8b63509b18d14717a0b979a40c7c6eaf as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/8b63509b18d14717a0b979a40c7c6eaf 2023-02-08 23:54:08,179 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/.tmp/cf1/97047d0d08a645b9aae7393c1d2b05c1 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/97047d0d08a645b9aae7393c1d2b05c1 2023-02-08 23:54:08,187 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 35b9c4ae42935ae148dc156df868a3bc/cf2 of 35b9c4ae42935ae148dc156df868a3bc into 8b63509b18d14717a0b979a40c7c6eaf(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:08,188 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 35b9c4ae42935ae148dc156df868a3bc: 2023-02-08 23:54:08,188 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc., storeName=35b9c4ae42935ae148dc156df868a3bc/cf2, priority=9, startTime=1675900448119; duration=0sec 2023-02-08 23:54:08,188 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:08,188 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 35b9c4ae42935ae148dc156df868a3bc/cf1 of 35b9c4ae42935ae148dc156df868a3bc into 97047d0d08a645b9aae7393c1d2b05c1(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:08,189 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 35b9c4ae42935ae148dc156df868a3bc: 2023-02-08 23:54:08,189 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc., storeName=35b9c4ae42935ae148dc156df868a3bc/cf1, priority=9, startTime=1675900448117; duration=0sec 2023-02-08 23:54:08,189 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:08,300 DEBUG [Listener at localhost/37973] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-08 23:54:08,311 INFO [Listener at localhost/37973] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-08 23:54:08,313 DEBUG [Listener at localhost/37973] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-08 23:54:08,346 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:08,347 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,347 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,358 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,358 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,359 INFO [Listener at localhost/37973] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-08 23:54:08,365 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-08 23:54:08,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:08,380 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900448380"}]},"ts":"1675900448380"} 2023-02-08 23:54:08,382 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-08 23:54:08,384 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,384 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,384 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-08 23:54:08,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 23:54:08,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5981761d0d9fef4c60f7e5990070c751, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=35b9c4ae42935ae148dc156df868a3bc, UNASSIGN}] 2023-02-08 23:54:08,389 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=35b9c4ae42935ae148dc156df868a3bc, UNASSIGN 2023-02-08 23:54:08,390 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=35b9c4ae42935ae148dc156df868a3bc, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:08,390 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900448390"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900448390"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900448390"}]},"ts":"1675900448390"} 2023-02-08 23:54:08,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 35b9c4ae42935ae148dc156df868a3bc, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:08,399 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5981761d0d9fef4c60f7e5990070c751, UNASSIGN 2023-02-08 23:54:08,401 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=5981761d0d9fef4c60f7e5990070c751, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:08,401 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900448400"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900448400"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900448400"}]},"ts":"1675900448400"} 2023-02-08 23:54:08,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 5981761d0d9fef4c60f7e5990070c751, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:08,405 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:54:08,486 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-08 23:54:08,487 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-08 23:54:08,490 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,490 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 23:54:08,548 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 5981761d0d9fef4c60f7e5990070c751 2023-02-08 23:54:08,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 5981761d0d9fef4c60f7e5990070c751, disabling compactions & flushes 2023-02-08 23:54:08,548 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751. 2023-02-08 23:54:08,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751. 2023-02-08 23:54:08,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751. after waiting 0 ms 2023-02-08 23:54:08,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751. 2023-02-08 23:54:08,554 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/7cb46d4532a44eda87b1ca74d684c08e.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/7cb46d4532a44eda87b1ca74d684c08e-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/6ff65a65e21d41529a83b4d76c08aca3.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/6ff65a65e21d41529a83b4d76c08aca3-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/1e01050191d74f09b5dbc6d0c0873826.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/1e01050191d74f09b5dbc6d0c0873826-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/2155b6bd47974950b0ce4afc663c4f3f.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/2155b6bd47974950b0ce4afc663c4f3f-bottom] to archive 2023-02-08 23:54:08,556 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:08,562 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/7cb46d4532a44eda87b1ca74d684c08e.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/7cb46d4532a44eda87b1ca74d684c08e.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,563 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/6ff65a65e21d41529a83b4d76c08aca3.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/6ff65a65e21d41529a83b4d76c08aca3.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,565 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/1e01050191d74f09b5dbc6d0c0873826.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/1e01050191d74f09b5dbc6d0c0873826.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,567 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/2155b6bd47974950b0ce4afc663c4f3f.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/2155b6bd47974950b0ce4afc663c4f3f.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,595 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/491d1ea3e14749ebb567f550bee27072.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/491d1ea3e14749ebb567f550bee27072-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/557d968cedac4aba92266d7d96b46e73.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/557d968cedac4aba92266d7d96b46e73-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/ba29610f803a44398a6f0e1eec52390c.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ba29610f803a44398a6f0e1eec52390c-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/c9c0dec0d6b94ddf901872809a0343fa.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/c9c0dec0d6b94ddf901872809a0343fa-bottom] to archive 2023-02-08 23:54:08,596 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:08,598 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/491d1ea3e14749ebb567f550bee27072.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/491d1ea3e14749ebb567f550bee27072.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,601 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/557d968cedac4aba92266d7d96b46e73.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/557d968cedac4aba92266d7d96b46e73.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,603 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/ba29610f803a44398a6f0e1eec52390c.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/ba29610f803a44398a6f0e1eec52390c.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,605 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/c9c0dec0d6b94ddf901872809a0343fa.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/c9c0dec0d6b94ddf901872809a0343fa.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 23:54:08,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:08,613 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751. 2023-02-08 23:54:08,613 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 5981761d0d9fef4c60f7e5990070c751: Waiting for close lock at 1675900448548Running coprocessor pre-close hooks at 1675900448548Disabling compacts and flushes for region at 1675900448548Disabling writes for close at 1675900448549 (+1 ms)Writing region close event to WAL at 1675900448606 (+57 ms)Running coprocessor post-close hooks at 1675900448612 (+6 ms)Closed at 1675900448613 (+1 ms) 2023-02-08 23:54:08,618 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=5981761d0d9fef4c60f7e5990070c751, regionState=CLOSED 2023-02-08 23:54:08,618 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 5981761d0d9fef4c60f7e5990070c751 2023-02-08 23:54:08,618 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:08,618 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900448618"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900448618"}]},"ts":"1675900448618"} 2023-02-08 23:54:08,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 35b9c4ae42935ae148dc156df868a3bc, disabling compactions & flushes 2023-02-08 23:54:08,619 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc. 2023-02-08 23:54:08,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc. 2023-02-08 23:54:08,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc. after waiting 0 ms 2023-02-08 23:54:08,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc. 2023-02-08 23:54:08,621 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/7cb46d4532a44eda87b1ca74d684c08e.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/7cb46d4532a44eda87b1ca74d684c08e-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/6ff65a65e21d41529a83b4d76c08aca3.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/6ff65a65e21d41529a83b4d76c08aca3-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/1e01050191d74f09b5dbc6d0c0873826.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/1e01050191d74f09b5dbc6d0c0873826-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/2155b6bd47974950b0ce4afc663c4f3f.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/2155b6bd47974950b0ce4afc663c4f3f-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-8fc6b0350c1744baac8be7dd29977be0, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-20c8043f893b45838a512a3d97be16ba, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-b635daea7a254c7cbb9341b3b5537aa4] to archive 2023-02-08 23:54:08,623 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:08,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-08 23:54:08,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 5981761d0d9fef4c60f7e5990070c751, server=jenkins-hbase4.apache.org,36199,1675900441120 in 218 msec 2023-02-08 23:54:08,626 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/7cb46d4532a44eda87b1ca74d684c08e.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/7cb46d4532a44eda87b1ca74d684c08e.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,627 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5981761d0d9fef4c60f7e5990070c751, UNASSIGN in 237 msec 2023-02-08 23:54:08,629 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/6ff65a65e21d41529a83b4d76c08aca3.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/6ff65a65e21d41529a83b4d76c08aca3.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,634 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/1e01050191d74f09b5dbc6d0c0873826.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/1e01050191d74f09b5dbc6d0c0873826.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,636 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/2155b6bd47974950b0ce4afc663c4f3f.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/2155b6bd47974950b0ce4afc663c4f3f.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,638 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-8fc6b0350c1744baac8be7dd29977be0 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-8fc6b0350c1744baac8be7dd29977be0 2023-02-08 23:54:08,640 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-20c8043f893b45838a512a3d97be16ba to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-20c8043f893b45838a512a3d97be16ba 2023-02-08 23:54:08,642 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-b635daea7a254c7cbb9341b3b5537aa4 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-b635daea7a254c7cbb9341b3b5537aa4 2023-02-08 23:54:08,645 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/491d1ea3e14749ebb567f550bee27072.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/491d1ea3e14749ebb567f550bee27072-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/557d968cedac4aba92266d7d96b46e73.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/557d968cedac4aba92266d7d96b46e73-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/ba29610f803a44398a6f0e1eec52390c.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ba29610f803a44398a6f0e1eec52390c-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/c9c0dec0d6b94ddf901872809a0343fa.952532b0193ae630451f659ecaa3180f->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/c9c0dec0d6b94ddf901872809a0343fa-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-edb7db88f93649d7b79caecab104fc6d, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-0f9fe5fb95b047d5b0bd7a7d920c7169, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-ee6b8b3d7084428fb4201ff37c7606a2] to archive 2023-02-08 23:54:08,646 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:08,648 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/491d1ea3e14749ebb567f550bee27072.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/491d1ea3e14749ebb567f550bee27072.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,650 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/557d968cedac4aba92266d7d96b46e73.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/557d968cedac4aba92266d7d96b46e73.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,652 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/ba29610f803a44398a6f0e1eec52390c.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/ba29610f803a44398a6f0e1eec52390c.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,654 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/c9c0dec0d6b94ddf901872809a0343fa.952532b0193ae630451f659ecaa3180f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/c9c0dec0d6b94ddf901872809a0343fa.952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,656 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-edb7db88f93649d7b79caecab104fc6d to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-edb7db88f93649d7b79caecab104fc6d 2023-02-08 23:54:08,658 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-0f9fe5fb95b047d5b0bd7a7d920c7169 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-0f9fe5fb95b047d5b0bd7a7d920c7169 2023-02-08 23:54:08,660 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-ee6b8b3d7084428fb4201ff37c7606a2 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/testSplitTableRegionUnevenDaughter=952532b0193ae630451f659ecaa3180f-ee6b8b3d7084428fb4201ff37c7606a2 2023-02-08 23:54:08,665 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 23:54:08,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:08,667 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc. 2023-02-08 23:54:08,667 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 35b9c4ae42935ae148dc156df868a3bc: Waiting for close lock at 1675900448618Running coprocessor pre-close hooks at 1675900448618Disabling compacts and flushes for region at 1675900448618Disabling writes for close at 1675900448619 (+1 ms)Writing region close event to WAL at 1675900448661 (+42 ms)Running coprocessor post-close hooks at 1675900448666 (+5 ms)Closed at 1675900448667 (+1 ms) 2023-02-08 23:54:08,670 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:08,671 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=35b9c4ae42935ae148dc156df868a3bc, regionState=CLOSED 2023-02-08 23:54:08,671 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900448671"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900448671"}]},"ts":"1675900448671"} 2023-02-08 23:54:08,676 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-08 23:54:08,676 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 35b9c4ae42935ae148dc156df868a3bc, server=jenkins-hbase4.apache.org,36199,1675900441120 in 279 msec 2023-02-08 23:54:08,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-02-08 23:54:08,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=35b9c4ae42935ae148dc156df868a3bc, UNASSIGN in 289 msec 2023-02-08 23:54:08,680 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900448680"}]},"ts":"1675900448680"} 2023-02-08 23:54:08,682 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-08 23:54:08,684 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-08 23:54:08,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 318 msec 2023-02-08 23:54:08,692 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,692 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 23:54:08,694 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-08 23:54:08,694 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,695 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,700 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-08 23:54:08,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:08,709 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:08,711 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:08,711 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,711 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-08 23:54:08,719 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,719 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751 2023-02-08 23:54:08,720 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:08,724 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/recovered.edits] 2023-02-08 23:54:08,724 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/recovered.edits] 2023-02-08 23:54:08,724 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/recovered.edits] 2023-02-08 23:54:08,728 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/97047d0d08a645b9aae7393c1d2b05c1 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf1/97047d0d08a645b9aae7393c1d2b05c1 2023-02-08 23:54:08,728 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/311b9e2d0af24caab497c8c4c1cefe56 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf1/311b9e2d0af24caab497c8c4c1cefe56 2023-02-08 23:54:08,728 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/1e01050191d74f09b5dbc6d0c0873826 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/1e01050191d74f09b5dbc6d0c0873826 2023-02-08 23:54:08,731 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/20c8043f893b45838a512a3d97be16ba to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/20c8043f893b45838a512a3d97be16ba 2023-02-08 23:54:08,733 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/2155b6bd47974950b0ce4afc663c4f3f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/2155b6bd47974950b0ce4afc663c4f3f 2023-02-08 23:54:08,733 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/8b63509b18d14717a0b979a40c7c6eaf to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/cf2/8b63509b18d14717a0b979a40c7c6eaf 2023-02-08 23:54:08,734 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/7c56875488d04069851e8143178bbb25 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/cf2/7c56875488d04069851e8143178bbb25 2023-02-08 23:54:08,735 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/6ff65a65e21d41529a83b4d76c08aca3 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/6ff65a65e21d41529a83b4d76c08aca3 2023-02-08 23:54:08,737 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/7cb46d4532a44eda87b1ca74d684c08e to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/7cb46d4532a44eda87b1ca74d684c08e 2023-02-08 23:54:08,738 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/recovered.edits/90.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc/recovered.edits/90.seqid 2023-02-08 23:54:08,738 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/recovered.edits/90.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751/recovered.edits/90.seqid 2023-02-08 23:54:08,739 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/8fc6b0350c1744baac8be7dd29977be0 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/8fc6b0350c1744baac8be7dd29977be0 2023-02-08 23:54:08,740 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/5981761d0d9fef4c60f7e5990070c751 2023-02-08 23:54:08,740 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:08,741 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/b635daea7a254c7cbb9341b3b5537aa4 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf1/b635daea7a254c7cbb9341b3b5537aa4 2023-02-08 23:54:08,745 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/0f9fe5fb95b047d5b0bd7a7d920c7169 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/0f9fe5fb95b047d5b0bd7a7d920c7169 2023-02-08 23:54:08,747 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/491d1ea3e14749ebb567f550bee27072 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/491d1ea3e14749ebb567f550bee27072 2023-02-08 23:54:08,749 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/557d968cedac4aba92266d7d96b46e73 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/557d968cedac4aba92266d7d96b46e73 2023-02-08 23:54:08,750 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ba29610f803a44398a6f0e1eec52390c to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ba29610f803a44398a6f0e1eec52390c 2023-02-08 23:54:08,752 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/c9c0dec0d6b94ddf901872809a0343fa to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/c9c0dec0d6b94ddf901872809a0343fa 2023-02-08 23:54:08,754 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/edb7db88f93649d7b79caecab104fc6d to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/edb7db88f93649d7b79caecab104fc6d 2023-02-08 23:54:08,755 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ee6b8b3d7084428fb4201ff37c7606a2 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/cf2/ee6b8b3d7084428fb4201ff37c7606a2 2023-02-08 23:54:08,759 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/recovered.edits/85.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f/recovered.edits/85.seqid 2023-02-08 23:54:08,759 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionUnevenDaughter/952532b0193ae630451f659ecaa3180f 2023-02-08 23:54:08,759 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-08 23:54:08,762 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:08,787 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-08 23:54:08,813 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,814 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-08 23:54:08,833 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-08 23:54:08,835 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:08,835 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-08 23:54:08,835 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900448835"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:08,836 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900448835"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:08,836 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900448835"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:08,839 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 23:54:08,839 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 952532b0193ae630451f659ecaa3180f, NAME => 'testSplitTableRegionUnevenDaughter,,1675900444028.952532b0193ae630451f659ecaa3180f.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 5981761d0d9fef4c60f7e5990070c751, NAME => 'testSplitTableRegionUnevenDaughter,,1675900446149.5981761d0d9fef4c60f7e5990070c751.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 35b9c4ae42935ae148dc156df868a3bc, NAME => 'testSplitTableRegionUnevenDaughter,26,1675900446149.35b9c4ae42935ae148dc156df868a3bc.', STARTKEY => '26', ENDKEY => ''}] 2023-02-08 23:54:08,839 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-08 23:54:08,839 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675900448839"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:08,841 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-08 23:54:08,844 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:08,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 143 msec 2023-02-08 23:54:09,015 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:09,015 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:09,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-08 23:54:09,017 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-08 23:54:09,034 INFO [Listener at localhost/37973] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=390 (was 360) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:39368 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:59128 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49c176f2-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:39338 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49c176f2-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49c176f2-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:36199-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49c176f2-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:59164 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:38996 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:59154 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:59112 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:39316 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:38972 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:39326 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:38048 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:58790 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:59138 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:39354 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49c176f2-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:39122 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:51006 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=853 (was 696) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=287 (was 286) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3622 (was 3671) 2023-02-08 23:54:09,048 INFO [Listener at localhost/37973] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=390, OpenFileDescriptor=853, MaxFileDescriptor=60000, SystemLoadAverage=287, ProcessCount=168, AvailableMemoryMB=3621 2023-02-08 23:54:09,049 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:09,049 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:09,049 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:09,054 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 23:54:09,055 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 23:54:09,056 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:09,058 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:54:09,059 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:54:09,061 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:09,061 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df empty. 2023-02-08 23:54:09,062 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:09,062 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-08 23:54:09,073 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:09,075 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => af07d15277922930e96dbc9b857cb0df, NAME => 'testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp 2023-02-08 23:54:09,087 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:09,087 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing af07d15277922930e96dbc9b857cb0df, disabling compactions & flushes 2023-02-08 23:54:09,087 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:09,087 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:09,087 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. after waiting 0 ms 2023-02-08 23:54:09,087 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:09,087 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:09,087 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for af07d15277922930e96dbc9b857cb0df: Waiting for close lock at 1675900449087Disabling compacts and flushes for region at 1675900449087Disabling writes for close at 1675900449087Writing region close event to WAL at 1675900449087Closed at 1675900449087 2023-02-08 23:54:09,090 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:54:09,091 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900449090"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900449090"}]},"ts":"1675900449090"} 2023-02-08 23:54:09,092 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:54:09,093 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:54:09,094 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900449094"}]},"ts":"1675900449094"} 2023-02-08 23:54:09,095 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-08 23:54:09,099 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:54:09,100 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:54:09,101 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:54:09,101 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:54:09,101 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 23:54:09,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, ASSIGN}] 2023-02-08 23:54:09,103 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, ASSIGN 2023-02-08 23:54:09,104 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42343,1675900441151; forceNewPlan=false, retain=false 2023-02-08 23:54:09,254 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:54:09,255 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=af07d15277922930e96dbc9b857cb0df, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,256 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900449255"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900449255"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900449255"}]},"ts":"1675900449255"} 2023-02-08 23:54:09,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:54:09,411 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,411 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 23:54:09,415 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34092, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 23:54:09,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:09,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => af07d15277922930e96dbc9b857cb0df, NAME => 'testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:09,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:09,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:09,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:09,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:09,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:09,421 INFO [StoreOpener-af07d15277922930e96dbc9b857cb0df-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:09,422 DEBUG [StoreOpener-af07d15277922930e96dbc9b857cb0df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1 2023-02-08 23:54:09,422 DEBUG [StoreOpener-af07d15277922930e96dbc9b857cb0df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1 2023-02-08 23:54:09,423 INFO [StoreOpener-af07d15277922930e96dbc9b857cb0df-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region af07d15277922930e96dbc9b857cb0df columnFamilyName cf1 2023-02-08 23:54:09,423 INFO [StoreOpener-af07d15277922930e96dbc9b857cb0df-1] regionserver.HStore(310): Store=af07d15277922930e96dbc9b857cb0df/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:09,423 INFO [StoreOpener-af07d15277922930e96dbc9b857cb0df-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:09,425 DEBUG [StoreOpener-af07d15277922930e96dbc9b857cb0df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2 2023-02-08 23:54:09,425 DEBUG [StoreOpener-af07d15277922930e96dbc9b857cb0df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2 2023-02-08 23:54:09,426 INFO [StoreOpener-af07d15277922930e96dbc9b857cb0df-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region af07d15277922930e96dbc9b857cb0df columnFamilyName cf2 2023-02-08 23:54:09,427 INFO [StoreOpener-af07d15277922930e96dbc9b857cb0df-1] regionserver.HStore(310): Store=af07d15277922930e96dbc9b857cb0df/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:09,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:09,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:09,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:09,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:09,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:09,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened af07d15277922930e96dbc9b857cb0df; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67936507, jitterRate=0.012332841753959656}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:09,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for af07d15277922930e96dbc9b857cb0df: Running coprocessor pre-open hook at 1675900449419Writing region info on filesystem at 1675900449419Initializing all the Stores at 1675900449420 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900449420Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900449421 (+1 ms)Cleaning up temporary data from old regions at 1675900449429 (+8 ms)Cleaning up detritus from prior splits at 1675900449429Running coprocessor post-open hooks at 1675900449435 (+6 ms)Region opened successfully at 1675900449435 2023-02-08 23:54:09,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df., pid=27, masterSystemTime=1675900449411 2023-02-08 23:54:09,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:09,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:09,442 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=af07d15277922930e96dbc9b857cb0df, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,442 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900449442"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900449442"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900449442"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900449442"}]},"ts":"1675900449442"} 2023-02-08 23:54:09,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-08 23:54:09,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 in 186 msec 2023-02-08 23:54:09,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-08 23:54:09,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, ASSIGN in 346 msec 2023-02-08 23:54:09,451 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:54:09,451 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900449451"}]},"ts":"1675900449451"} 2023-02-08 23:54:09,453 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-08 23:54:09,455 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:54:09,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 400 msec 2023-02-08 23:54:09,560 DEBUG [Listener at localhost/37973] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:54:09,564 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34096, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:54:09,571 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:09,571 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:09,572 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:09,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 23:54:09,574 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 23:54:09,574 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,574 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 23:54:09,574 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 23:54:09,575 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,575 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,577 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,577 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,577 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,577 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,577 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,578 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:09,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:09,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:09,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:09,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,579 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:09,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,579 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:09,579 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:09,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:09,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:09,579 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:09,579 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:09,579 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,579 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,580 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:09,581 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,581 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:09,582 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:09,582 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:09,582 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:09,582 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:09,582 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:09,582 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. started... 2023-02-08 23:54:09,582 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:09,583 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing af07d15277922930e96dbc9b857cb0df 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 23:54:09,583 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,583 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,583 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:09,585 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,585 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,585 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:09,599 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/331696e6c7f44124ab8dfc6480ff0507 2023-02-08 23:54:09,617 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/17ce6af538d441f4aef2dc6d6dacff6f 2023-02-08 23:54:09,623 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/331696e6c7f44124ab8dfc6480ff0507 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507 2023-02-08 23:54:09,628 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:54:09,628 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/17ce6af538d441f4aef2dc6d6dacff6f as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f 2023-02-08 23:54:09,634 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:54:09,635 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for af07d15277922930e96dbc9b857cb0df in 52ms, sequenceid=6, compaction requested=false 2023-02-08 23:54:09,635 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-08 23:54:09,636 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for af07d15277922930e96dbc9b857cb0df: 2023-02-08 23:54:09,636 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:09,636 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:09,636 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:09,636 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,636 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:09,636 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:09,637 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,638 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:09,638 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,638 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:09,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 23:54:09,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ec9c728[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:09,640 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 23:54:09,640 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,642 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,642 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,642 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,642 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:09,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,643 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,643 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:09,643 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:09,643 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:09,642 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:09,643 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:09,643 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:09,643 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:09,643 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:09,643 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:09,644 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,644 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:09,644 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:09,644 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:09,645 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:09,645 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:09,645 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:09,646 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:09,645 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:09,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:09,646 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:09,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:09,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:09,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:09,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:09,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:54:09,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 23:54:09,650 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 23:54:09,650 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:09,650 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:09,650 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 23:54:09,650 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:09,652 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,652 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,652 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:09,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:09,652 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,652 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,652 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,652 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:09,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,653 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:09,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,654 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:09,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,654 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:09,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:09,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:09,656 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,656 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,657 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,660 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,661 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,661 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,661 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,661 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,661 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,661 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:09,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,661 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,661 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,662 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,661 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 23:54:09,662 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:09,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,662 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,662 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,663 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:09,663 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:09,663 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,663 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,663 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,663 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,663 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,663 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,763 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:09,763 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:09,763 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:09,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:09,783 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:09,784 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:09,784 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:09,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:09,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 23:54:09,787 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 23:54:09,787 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,787 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 23:54:09,787 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 23:54:09,788 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,788 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,790 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,790 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,790 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,790 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,791 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,791 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:09,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,791 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,792 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:09,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,792 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:09,792 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:09,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:09,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,792 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:09,792 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:09,792 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,793 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:09,794 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:09,794 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:09,794 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,794 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:09,795 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:09,795 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:09,794 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:09,795 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:09,795 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:09,795 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. started... 2023-02-08 23:54:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,795 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing af07d15277922930e96dbc9b857cb0df 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:09,795 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,796 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:09,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,797 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:09,797 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:09,797 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:09,797 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:09,797 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,797 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:09,797 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,798 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:09,798 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:09,798 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:09,801 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,801 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,801 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:09,811 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/3c19d9bcc3534afd87e2766e56865326 2023-02-08 23:54:09,826 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/b69b10c7ba0542be8bd09b0826df7849 2023-02-08 23:54:09,832 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/3c19d9bcc3534afd87e2766e56865326 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326 2023-02-08 23:54:09,837 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:54:09,838 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/b69b10c7ba0542be8bd09b0826df7849 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849 2023-02-08 23:54:09,842 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:54:09,843 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for af07d15277922930e96dbc9b857cb0df in 48ms, sequenceid=19, compaction requested=false 2023-02-08 23:54:09,843 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for af07d15277922930e96dbc9b857cb0df: 2023-02-08 23:54:09,843 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:09,843 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:09,843 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:09,843 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,843 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:09,843 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:09,845 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,845 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:09,845 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,846 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:09,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:09,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:09,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:09,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:09,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 23:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8abf65b[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:09,848 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 23:54:09,848 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,849 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,849 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,849 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:09,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:09,849 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,850 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:09,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:09,854 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:09,854 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:09,854 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:09,854 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:09,854 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:09,855 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:09,856 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:09,856 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,856 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:09,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:09,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:09,856 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:09,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:09,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:09,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:09,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:09,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:09,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:54:09,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 23:54:09,860 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 23:54:09,860 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:09,860 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:09,860 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 23:54:09,860 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:09,862 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,862 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,862 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,862 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,862 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,862 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:09,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:09,862 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:09,862 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,863 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:09,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,863 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:09,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:09,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:09,865 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:09,868 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,868 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,872 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,872 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,872 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,872 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,872 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,872 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:09,872 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,872 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 23:54:09,873 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:09,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,872 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,872 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,873 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,873 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,873 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:09,873 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,874 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:09,874 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,874 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:09,874 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,874 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:09,874 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,874 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:09,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,974 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:09,974 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:09,975 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:09,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:09,993 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:09,993 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:09,993 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:09,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:09,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 23:54:09,995 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 23:54:09,995 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,996 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 23:54:09,996 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 23:54:09,997 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:09,997 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,998 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,998 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,998 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,998 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,999 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:09,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,999 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:09,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:09,999 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,000 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:10,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,000 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:10,000 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:10,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,000 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:10,000 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:10,000 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:10,001 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,000 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:10,001 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:10,002 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:10,002 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:10,002 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:10,002 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:10,002 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:10,002 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,002 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,003 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:10,002 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:10,004 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:10,003 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:10,003 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:10,004 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:10,004 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:10,003 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:10,004 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:10,004 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. started... 2023-02-08 23:54:10,004 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing af07d15277922930e96dbc9b857cb0df 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:10,006 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,006 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,006 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,006 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,007 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,007 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,016 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/86a819126854495c8db28c148ce0456c 2023-02-08 23:54:10,031 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/cd4287af543c4f2091447436b58f328a 2023-02-08 23:54:10,036 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/86a819126854495c8db28c148ce0456c as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c 2023-02-08 23:54:10,042 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:54:10,043 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/cd4287af543c4f2091447436b58f328a as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a 2023-02-08 23:54:10,048 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:54:10,050 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for af07d15277922930e96dbc9b857cb0df in 45ms, sequenceid=32, compaction requested=true 2023-02-08 23:54:10,050 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for af07d15277922930e96dbc9b857cb0df: 2023-02-08 23:54:10,050 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:10,050 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:10,050 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:10,050 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,050 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:10,050 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:10,051 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,051 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,052 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,052 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 23:54:10,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ac0168f[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:10,054 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 23:54:10,054 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,056 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,056 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,056 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,056 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,057 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:10,057 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:10,057 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:10,056 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:10,057 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:10,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,057 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:10,057 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:10,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,057 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:10,057 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:10,057 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:10,058 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:10,058 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:10,059 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,058 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,059 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:10,059 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:10,060 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,060 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:10,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,060 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,061 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:10,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:54:10,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 23:54:10,064 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 23:54:10,064 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:10,064 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:10,065 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 23:54:10,065 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:10,066 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,066 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,066 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,066 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,066 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,066 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,067 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,066 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,067 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,068 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,069 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,070 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,070 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,073 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-08 23:54:10,073 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-08 23:54:10,074 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,074 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,074 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:10,074 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,074 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,074 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,074 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-08 23:54:10,074 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,074 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 23:54:10,075 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,074 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,074 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,075 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,076 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,075 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:10,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,076 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,076 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitWithoutPONR because node does not exist (not an error) 2023-02-08 23:54:10,076 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,076 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,076 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,076 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,076 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,076 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:10,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,076 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:10,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,177 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:10,177 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:10,177 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:10,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:10,195 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:10,195 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:10,196 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:10,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:10,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 23:54:10,198 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 23:54:10,198 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,199 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 23:54:10,199 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 23:54:10,199 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,199 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,200 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,200 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,200 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,200 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,201 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,201 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:10,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,201 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,201 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,202 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:10,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,202 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:10,202 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:10,202 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:10,202 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:10,202 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:10,202 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:10,204 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:10,202 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:10,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:10,205 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:10,205 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:10,205 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,205 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,205 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:10,206 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:10,206 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:10,205 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,206 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:10,207 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:10,206 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:10,208 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:10,208 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:10,208 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:10,208 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:10,208 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. started... 2023-02-08 23:54:10,208 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing af07d15277922930e96dbc9b857cb0df 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:10,211 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,212 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,212 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,213 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,213 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,213 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,219 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/3aa70ba641eb4587b40ee0c0e7892101 2023-02-08 23:54:10,233 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4 2023-02-08 23:54:10,239 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/3aa70ba641eb4587b40ee0c0e7892101 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101 2023-02-08 23:54:10,244 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:54:10,244 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4 2023-02-08 23:54:10,249 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:54:10,250 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for af07d15277922930e96dbc9b857cb0df in 42ms, sequenceid=45, compaction requested=true 2023-02-08 23:54:10,250 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for af07d15277922930e96dbc9b857cb0df: 2023-02-08 23:54:10,250 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:10,251 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:10,251 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:10,251 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,251 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:10,251 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:10,252 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,252 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,253 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,253 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 23:54:10,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f3fac9f[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:10,255 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 23:54:10,255 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,256 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,256 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,256 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,256 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:10,256 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:10,256 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:10,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,256 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:10,256 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:10,256 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:10,256 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,257 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,257 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:10,257 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:10,257 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:10,258 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:10,258 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,258 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:10,259 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,261 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:10,261 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,261 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:10,262 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:10,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,262 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,262 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:10,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:54:10,267 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 23:54:10,267 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 23:54:10,267 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:10,267 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:10,267 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 23:54:10,267 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:10,269 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,269 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,269 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,269 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,269 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,269 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,269 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,269 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,270 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,271 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,272 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,273 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,273 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,276 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,276 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,276 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,276 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,276 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,276 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,276 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,276 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,276 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,276 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,276 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,276 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 23:54:10,277 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,276 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,277 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,277 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,277 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:10,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,277 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:10,277 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,278 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,278 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,278 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,378 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:10,378 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:10,378 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:10,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:10,399 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:10,399 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:10,400 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:10,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:10,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 23:54:10,402 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 23:54:10,402 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,403 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 23:54:10,403 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 23:54:10,403 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,403 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,406 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,406 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,406 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,406 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,407 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,407 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:10,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,407 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:10,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,408 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:10,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,408 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:10,408 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:10,408 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:10,408 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:10,408 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:10,408 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:10,408 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:10,408 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,408 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:10,408 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:10,408 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:10,408 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,408 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,408 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:10,408 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:10,408 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:10,409 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:10,408 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:10,409 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:10,409 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:10,409 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:10,409 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:10,409 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. started... 2023-02-08 23:54:10,409 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:10,409 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing af07d15277922930e96dbc9b857cb0df 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:10,410 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,411 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,411 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,411 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,412 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,412 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,422 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/6a77b2b4cce440d89ecab656c0e50559 2023-02-08 23:54:10,436 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/a723a1914a244cd880e78894b22ee576 2023-02-08 23:54:10,442 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/6a77b2b4cce440d89ecab656c0e50559 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559 2023-02-08 23:54:10,447 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:54:10,448 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/a723a1914a244cd880e78894b22ee576 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576 2023-02-08 23:54:10,453 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:54:10,454 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for af07d15277922930e96dbc9b857cb0df in 45ms, sequenceid=58, compaction requested=true 2023-02-08 23:54:10,454 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for af07d15277922930e96dbc9b857cb0df: 2023-02-08 23:54:10,454 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:10,454 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:10,454 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:10,454 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,454 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:10,454 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:10,455 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,455 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,456 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,456 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 23:54:10,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f8fced7[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:10,458 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 23:54:10,458 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,460 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,460 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,460 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,460 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,461 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:10,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,460 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:10,461 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:10,461 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:10,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,461 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:10,461 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:10,461 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:10,461 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:10,461 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:10,461 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:10,462 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:10,462 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,462 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:10,463 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:10,463 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,463 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:10,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,463 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,463 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,463 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:10,463 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:10,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:54:10,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 23:54:10,467 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 23:54:10,467 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:10,467 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:10,467 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 23:54:10,467 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:10,468 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,468 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,468 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,468 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,468 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,468 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,468 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,469 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,469 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,470 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,471 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,472 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,472 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,477 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,477 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,477 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,477 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,477 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,477 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,477 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,477 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,477 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,477 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,477 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 23:54:10,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,477 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,478 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:10,478 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,478 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,478 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,478 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,478 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,478 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:10,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,479 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:10,478 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,479 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,579 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:10,579 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:10,579 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:10,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:10,597 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:10,597 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:10,597 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:10,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:10,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 23:54:10,599 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 23:54:10,599 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,599 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 23:54:10,599 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 23:54:10,600 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,600 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,601 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,601 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,601 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,601 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,602 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,602 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:10,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,603 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:10,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,603 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:10,603 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 23:54:10,603 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:10,603 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:10,603 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:10,603 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:10,603 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:10,603 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 23:54:10,603 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,603 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,603 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 23:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 23:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 23:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:10,604 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:10,604 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. started... 2023-02-08 23:54:10,604 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing af07d15277922930e96dbc9b857cb0df 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:10,605 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,606 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,606 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,606 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,606 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,606 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,616 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/26db8435edd1481db01a2586f88d3bc0 2023-02-08 23:54:10,632 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/fe147b97248a4ca4b8420ea4832f9c05 2023-02-08 23:54:10,637 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/26db8435edd1481db01a2586f88d3bc0 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0 2023-02-08 23:54:10,642 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:54:10,643 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/fe147b97248a4ca4b8420ea4832f9c05 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05 2023-02-08 23:54:10,648 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:54:10,649 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for af07d15277922930e96dbc9b857cb0df in 45ms, sequenceid=71, compaction requested=true 2023-02-08 23:54:10,649 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for af07d15277922930e96dbc9b857cb0df: 2023-02-08 23:54:10,649 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:10,649 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:10,650 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:10,650 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,650 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 23:54:10,650 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 23:54:10,652 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,652 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,653 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,653 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 23:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38c1e453[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:10,654 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 23:54:10,655 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,656 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,656 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,656 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,656 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:10,656 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,656 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:10,656 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,656 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:10,657 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:10,657 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:10,656 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,657 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 23:54:10,657 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 23:54:10,657 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:10,658 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,658 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:10,658 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,658 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:10,659 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:10,659 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 23:54:10,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,659 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,659 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,659 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:10,659 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 23:54:10,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:54:10,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 23:54:10,662 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 23:54:10,662 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:10,662 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:10,662 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 23:54:10,662 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:10,665 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,665 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,665 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,665 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,665 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,665 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,665 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,665 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,666 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,666 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,668 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,669 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,669 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 23:54:10,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,671 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,671 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,671 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,671 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,671 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,672 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,672 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,672 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,672 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,672 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,672 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 23:54:10,672 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:10,672 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,672 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 23:54:10,673 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:10,673 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:10,673 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:10,673 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,673 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:10,673 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:10,673 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 23:54:10,673 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 23:54:10,773 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:10,773 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:10,773 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:10,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:11,037 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-08 23:54:11,038 DEBUG [Listener at localhost/37973] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:11,040 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,041 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,041 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,041 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,042 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:11,042 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:11,042 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:11,290 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,291 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:11,291 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:11,291 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:11,293 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-08 23:54:11,294 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:11,294 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:11,294 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:11,294 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:11,295 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:11,299 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-08 23:54:11,301 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:11,301 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:11,302 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:11,302 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:11,303 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:11,303 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:11,304 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:11,304 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:11,305 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:11,305 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,306 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:11,306 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,306 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,306 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,307 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-08 23:54:11,307 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:11,308 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:11,309 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900481309 2023-02-08 23:54:11,311 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,311 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:11,316 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:11,316 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=af07d15277922930e96dbc9b857cb0df, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:54:11,316 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:11,316 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900511316 2023-02-08 23:54:11,316 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900571316 2023-02-08 23:54:11,316 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-08 23:54:11,317 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:11,319 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:11,319 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:11,320 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:11,567 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,567 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:11,567 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:11,567 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:11,567 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:11,567 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:11,567 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:11,567 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:11,567 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:11,568 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:11,570 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-08 23:54:11,570 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:11,571 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:11,571 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:11,571 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:11,571 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:11,572 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:11,572 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:11,572 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:11,573 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:11,573 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,573 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:11,573 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,573 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,573 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,574 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 23:54:11,574 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:11,575 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:11,575 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900481575 2023-02-08 23:54:11,575 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:11,577 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,579 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:11,580 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=af07d15277922930e96dbc9b857cb0df, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:54:11,580 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:11,580 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900511580 2023-02-08 23:54:11,580 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900571580 2023-02-08 23:54:11,580 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 23:54:11,580 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:11,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN}] 2023-02-08 23:54:11,582 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:11,582 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:11,582 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:11,830 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,830 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:11,830 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:11,830 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:11,831 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:11,831 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:11,831 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:11,831 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:11,831 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:11,832 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:11,833 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-08 23:54:11,834 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:11,834 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:11,834 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:11,835 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:11,835 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:11,836 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:11,836 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:11,836 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:11,837 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:11,837 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,837 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:11,837 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,838 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,838 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,838 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 23:54:11,838 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:11,839 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:11,839 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900481839 2023-02-08 23:54:11,840 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:11,841 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:11,844 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:11,844 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=af07d15277922930e96dbc9b857cb0df, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:54:11,844 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:11,844 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900511844 2023-02-08 23:54:11,844 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900571844 2023-02-08 23:54:11,845 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 23:54:11,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN}] 2023-02-08 23:54:11,845 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:11,847 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:11,849 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=af07d15277922930e96dbc9b857cb0df, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:11,849 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900451848"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900451848"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900451848"}]},"ts":"1675900451848"} 2023-02-08 23:54:11,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:54:11,851 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:11,851 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:11,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:11,851 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:12,095 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,095 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:12,095 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:12,095 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:12,096 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:12,096 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:12,096 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:12,096 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:12,097 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:12,097 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:12,099 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 23:54:12,099 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 23:54:12,099 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:12,100 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:12,100 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:12,100 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:12,101 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:12,101 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:12,101 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:12,101 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:12,102 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:12,103 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,103 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:12,103 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:12,103 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,103 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,103 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,103 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:12,103 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:12,103 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 23:54:12,104 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:12,104 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:12,105 INFO [Listener at localhost/37973] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df to restore RIT 2023-02-08 23:54:12,105 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900482105 2023-02-08 23:54:12,106 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:12,106 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:12,108 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:12,109 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=af07d15277922930e96dbc9b857cb0df, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:54:12,109 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:12,109 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900512109 2023-02-08 23:54:12,109 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900572109 2023-02-08 23:54:12,109 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:12,109 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=af07d15277922930e96dbc9b857cb0df, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:12,109 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900452109"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900452109"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900452109"}]},"ts":"1675900452109"} 2023-02-08 23:54:12,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:54:12,111 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:12,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing af07d15277922930e96dbc9b857cb0df, disabling compactions & flushes 2023-02-08 23:54:12,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:12,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:12,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. after waiting 0 ms 2023-02-08 23:54:12,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:12,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing af07d15277922930e96dbc9b857cb0df 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 23:54:12,277 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/05076176f46d4ccfafd9aaf5c9524aa9 2023-02-08 23:54:12,295 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/932d721e58884dd3b2b8823bf2ff21be 2023-02-08 23:54:12,301 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf1/05076176f46d4ccfafd9aaf5c9524aa9 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9 2023-02-08 23:54:12,306 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:54:12,307 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/.tmp/cf2/932d721e58884dd3b2b8823bf2ff21be as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be 2023-02-08 23:54:12,312 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:54:12,313 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for af07d15277922930e96dbc9b857cb0df in 48ms, sequenceid=82, compaction requested=true 2023-02-08 23:54:12,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 23:54:12,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:12,336 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. 2023-02-08 23:54:12,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for af07d15277922930e96dbc9b857cb0df: Waiting for close lock at 1675900452265Running coprocessor pre-close hooks at 1675900452265Disabling compacts and flushes for region at 1675900452265Disabling writes for close at 1675900452265Obtaining lock to block concurrent updates at 1675900452265Preparing flush snapshotting stores in af07d15277922930e96dbc9b857cb0df at 1675900452265Finished memstore snapshotting testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675900452266 (+1 ms)Flushing stores of testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. at 1675900452266Flushing af07d15277922930e96dbc9b857cb0df/cf1: creating writer at 1675900452267 (+1 ms)Flushing af07d15277922930e96dbc9b857cb0df/cf1: appending metadata at 1675900452269 (+2 ms)Flushing af07d15277922930e96dbc9b857cb0df/cf1: closing flushed file at 1675900452269Flushing af07d15277922930e96dbc9b857cb0df/cf2: creating writer at 1675900452283 (+14 ms)Flushing af07d15277922930e96dbc9b857cb0df/cf2: appending metadata at 1675900452285 (+2 ms)Flushing af07d15277922930e96dbc9b857cb0df/cf2: closing flushed file at 1675900452286 (+1 ms)Flushing af07d15277922930e96dbc9b857cb0df/cf1: reopening flushed file at 1675900452302 (+16 ms)Flushing af07d15277922930e96dbc9b857cb0df/cf2: reopening flushed file at 1675900452307 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for af07d15277922930e96dbc9b857cb0df in 48ms, sequenceid=82, compaction requested=true at 1675900452313 (+6 ms)Writing region close event to WAL at 1675900452331 (+18 ms)Running coprocessor post-close hooks at 1675900452336 (+5 ms)Closed at 1675900452336 2023-02-08 23:54:12,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,339 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=af07d15277922930e96dbc9b857cb0df, regionState=CLOSED 2023-02-08 23:54:12,339 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900452339"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900452339"}]},"ts":"1675900452339"} 2023-02-08 23:54:12,341 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:12,341 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:12,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:12,341 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:12,359 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,359 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:12,359 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:12,360 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:12,360 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:12,360 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:12,360 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:12,360 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:12,360 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:12,361 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:12,362 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 23:54:12,362 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-08 23:54:12,362 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-08 23:54:12,363 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:12,363 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:12,363 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:12,363 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:12,364 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:12,364 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:12,364 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:12,364 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:12,365 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:12,366 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,366 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:12,366 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:12,366 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:12,366 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,366 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,366 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,366 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:12,366 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:12,366 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:12,366 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 23:54:12,367 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:12,367 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:12,368 INFO [Listener at localhost/37973] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df to restore RIT 2023-02-08 23:54:12,368 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900482368 2023-02-08 23:54:12,369 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:12,369 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:12,371 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:12,372 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=af07d15277922930e96dbc9b857cb0df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:54:12,372 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:12,372 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900512372 2023-02-08 23:54:12,372 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900572372 2023-02-08 23:54:12,372 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:12,372 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=af07d15277922930e96dbc9b857cb0df, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:12,372 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900452372"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900452372"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900452372"}]},"ts":"1675900452372"} 2023-02-08 23:54:12,374 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:12,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-08 23:54:12,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 in 263 msec 2023-02-08 23:54:12,377 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:12,378 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:12,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:12,378 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:12,622 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,622 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:12,623 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:12,623 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:12,623 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:12,623 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:12,623 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:12,623 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:12,623 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:12,624 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:12,625 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 23:54:12,626 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-08 23:54:12,626 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:12,626 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:12,627 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:12,627 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:12,627 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:12,628 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:12,628 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:12,628 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:12,628 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:12,629 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:12,629 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,630 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:12,630 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:12,630 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:12,630 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,630 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,630 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,630 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:12,630 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:12,630 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 23:54:12,631 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:12,631 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:12,632 INFO [Listener at localhost/37973] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df to restore RIT 2023-02-08 23:54:12,632 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900482632 2023-02-08 23:54:12,633 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:12,633 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:12,635 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:12,636 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=af07d15277922930e96dbc9b857cb0df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:54:12,636 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:12,636 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900512636 2023-02-08 23:54:12,636 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900572636 2023-02-08 23:54:12,636 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:12,636 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:12,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:12,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-08 23:54:12,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN in 793 msec 2023-02-08 23:54:12,644 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:12,644 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:12,644 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:12,886 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,886 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:12,887 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:12,887 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:12,887 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:12,887 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:12,887 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:12,887 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:12,887 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:12,888 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:12,889 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 23:54:12,889 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:54:12,890 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:12,890 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:12,890 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:12,890 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:12,891 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:12,891 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:12,891 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:12,892 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:12,892 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:12,892 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:12,893 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,893 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:12,893 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:12,893 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:12,893 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:12,893 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:12,893 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 23:54:12,894 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:12,894 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:12,895 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900482895 2023-02-08 23:54:12,896 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:12,896 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,899 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:12,899 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=af07d15277922930e96dbc9b857cb0df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:54:12,899 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:12,899 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900512899 2023-02-08 23:54:12,899 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900572899 2023-02-08 23:54:12,900 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:12,900 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,900 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,900 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:12,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:12,916 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=af07d15277922930e96dbc9b857cb0df, threads=14 2023-02-08 23:54:12,917 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,917 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,917 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,918 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,918 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,918 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,918 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,918 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,918 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,918 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,918 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,919 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,919 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:12,919 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,059 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,063 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,072 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,083 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,095 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,097 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,099 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,106 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,106 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,106 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,109 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,110 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,110 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,110 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,110 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region af07d15277922930e96dbc9b857cb0df Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 23:54:13,155 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:13,155 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:13,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:13,155 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:13,342 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:54:13,411 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:13,411 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:13,411 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:13,411 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:13,413 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:13,415 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:13,415 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:13,415 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:13,415 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:13,417 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-08 23:54:13,418 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-08 23:54:13,418 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:54:13,418 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:13,419 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:13,420 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:13,420 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:13,421 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:13,421 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:13,421 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:13,423 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:13,427 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:13,428 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:13,429 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:13,429 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:13,429 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:13,429 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:13,430 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:13,430 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:13,430 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:13,430 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:13,430 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:13,430 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 13 msec 2023-02-08 23:54:13,431 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:13,432 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:13,436 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900483436 2023-02-08 23:54:13,437 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:13,437 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:13,440 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:13,440 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=af07d15277922930e96dbc9b857cb0df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:54:13,440 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:13,440 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900513440 2023-02-08 23:54:13,441 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900573441 2023-02-08 23:54:13,441 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:13,452 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=af07d15277922930e96dbc9b857cb0df, threads=14 2023-02-08 23:54:13,453 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,453 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,453 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,453 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,454 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,454 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,454 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,454 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,455 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,455 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,455 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,455 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,456 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,456 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,579 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,612 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,619 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,633 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,643 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,660 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,666 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,680 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,680 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,688 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,692 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,694 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:13,701 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507 for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:14,050 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a for region: af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:14,051 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region af07d15277922930e96dbc9b857cb0df Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 23:54:14,093 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:14,099 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 23:54:14,101 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 23:54:14,101 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:14,101 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:14,101 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:14,191 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,191 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:14,191 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:14,191 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:14,192 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:14,192 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:14,192 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:14,192 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:14,192 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:14,193 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:14,194 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-08 23:54:14,194 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:54:14,194 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:14,195 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:14,195 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:14,196 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:14,196 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:14,197 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:14,197 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:14,198 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:14,198 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:14,198 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:14,199 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,199 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:14,199 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:14,199 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:14,200 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,200 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,200 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,200 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:14,200 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:14,200 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-08 23:54:14,204 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:14,204 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:14,205 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900484205 2023-02-08 23:54:14,206 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:14,206 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,209 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:14,209 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=af07d15277922930e96dbc9b857cb0df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:54:14,209 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:14,209 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900514209 2023-02-08 23:54:14,209 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900574209 2023-02-08 23:54:14,209 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:14,212 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:14,214 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:14,214 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:14,215 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:14,459 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:14,459 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:14,460 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:14,460 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:14,461 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:14,461 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:14,461 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:14,461 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:14,461 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:14,462 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:14,463 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-08 23:54:14,463 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:54:14,464 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:14,464 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:14,464 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:14,465 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:14,465 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:14,465 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:14,466 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:14,466 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:14,466 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:14,467 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:14,467 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,468 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN 2023-02-08 23:54:14,468 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:14,468 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:14,468 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,468 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,468 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,468 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=af07d15277922930e96dbc9b857cb0df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:14,468 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure af07d15277922930e96dbc9b857cb0df, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:14,468 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 23:54:14,469 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:14,470 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:14,470 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900484470 2023-02-08 23:54:14,470 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:14,472 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:14,474 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:14,474 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=af07d15277922930e96dbc9b857cb0df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:54:14,474 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:14,474 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900514474 2023-02-08 23:54:14,474 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900574474 2023-02-08 23:54:14,474 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 23:54:14,477 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900454477"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675900454477"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675900454477"}]},"ts":"1675900454477"} 2023-02-08 23:54:14,477 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900454477"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900454477"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900454477"}]},"ts":"1675900454477"} 2023-02-08 23:54:14,477 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900454477"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900454477"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900454477"}]},"ts":"1675900454477"} 2023-02-08 23:54:14,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fe03d2fe0b831458763c25fce0549d32, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=335bec642c9991062859430c97058271, ASSIGN}] 2023-02-08 23:54:14,485 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=335bec642c9991062859430c97058271, ASSIGN 2023-02-08 23:54:14,486 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=335bec642c9991062859430c97058271, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42343,1675900441151; forceNewPlan=false, retain=false 2023-02-08 23:54:14,487 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fe03d2fe0b831458763c25fce0549d32, ASSIGN 2023-02-08 23:54:14,488 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fe03d2fe0b831458763c25fce0549d32, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42343,1675900441151; forceNewPlan=false, retain=false 2023-02-08 23:54:14,636 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 23:54:14,638 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=335bec642c9991062859430c97058271, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:14,638 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900454638"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900454638"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900454638"}]},"ts":"1675900454638"} 2023-02-08 23:54:14,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 335bec642c9991062859430c97058271, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:54:14,646 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=fe03d2fe0b831458763c25fce0549d32, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:14,646 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900454646"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900454646"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900454646"}]},"ts":"1675900454646"} 2023-02-08 23:54:14,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure fe03d2fe0b831458763c25fce0549d32, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:54:14,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271. 2023-02-08 23:54:14,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 335bec642c9991062859430c97058271, NAME => 'testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.', STARTKEY => '41', ENDKEY => ''} 2023-02-08 23:54:14,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:14,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 335bec642c9991062859430c97058271 2023-02-08 23:54:14,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:14,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 335bec642c9991062859430c97058271 2023-02-08 23:54:14,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 335bec642c9991062859430c97058271 2023-02-08 23:54:14,802 INFO [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 335bec642c9991062859430c97058271 2023-02-08 23:54:14,803 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1 2023-02-08 23:54:14,803 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1 2023-02-08 23:54:14,804 INFO [StoreOpener-335bec642c9991062859430c97058271-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 335bec642c9991062859430c97058271 columnFamilyName cf1 2023-02-08 23:54:14,822 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9-top 2023-02-08 23:54:14,827 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0-top 2023-02-08 23:54:14,832 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507-top 2023-02-08 23:54:14,839 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101-top 2023-02-08 23:54:14,844 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326-top 2023-02-08 23:54:14,849 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559-top 2023-02-08 23:54:14,857 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c-top 2023-02-08 23:54:14,857 INFO [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(310): Store=335bec642c9991062859430c97058271/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:14,857 INFO [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 335bec642c9991062859430c97058271 2023-02-08 23:54:14,858 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2 2023-02-08 23:54:14,858 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2 2023-02-08 23:54:14,859 INFO [StoreOpener-335bec642c9991062859430c97058271-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 335bec642c9991062859430c97058271 columnFamilyName cf2 2023-02-08 23:54:14,875 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f-top 2023-02-08 23:54:14,880 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be-top 2023-02-08 23:54:14,884 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576-top 2023-02-08 23:54:14,888 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849-top 2023-02-08 23:54:14,893 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a-top 2023-02-08 23:54:14,897 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4-top 2023-02-08 23:54:14,902 DEBUG [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05-top 2023-02-08 23:54:14,902 INFO [StoreOpener-335bec642c9991062859430c97058271-1] regionserver.HStore(310): Store=335bec642c9991062859430c97058271/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:14,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271 2023-02-08 23:54:14,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271 2023-02-08 23:54:14,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:14,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 335bec642c9991062859430c97058271 2023-02-08 23:54:14,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 335bec642c9991062859430c97058271; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71669174, jitterRate=0.0679539144039154}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:14,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 335bec642c9991062859430c97058271: Running coprocessor pre-open hook at 1675900454799Writing region info on filesystem at 1675900454799Initializing all the Stores at 1675900454800 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900454800Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900454802 (+2 ms)Cleaning up temporary data from old regions at 1675900454905 (+103 ms)Cleaning up detritus from prior splits at 1675900454905Running coprocessor post-open hooks at 1675900454908 (+3 ms)Region opened successfully at 1675900454908 2023-02-08 23:54:14,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271., pid=33, masterSystemTime=1675900454794 2023-02-08 23:54:14,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:14,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:54:14,910 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:54:14,910 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:54:14,910 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271. 2023-02-08 23:54:14,910 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271. 2023-02-08 23:54:14,910 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1912): 335bec642c9991062859430c97058271/cf1 is initiating minor compaction (all files) 2023-02-08 23:54:14,910 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1912): 335bec642c9991062859430c97058271/cf2 is initiating minor compaction (all files) 2023-02-08 23:54:14,910 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 335bec642c9991062859430c97058271/cf1 in testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271. 2023-02-08 23:54:14,910 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 335bec642c9991062859430c97058271/cf2 in testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271. 2023-02-08 23:54:14,911 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be-top] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/.tmp, totalSize=34.9 K 2023-02-08 23:54:14,911 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9-top] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/.tmp, totalSize=34.9 K 2023-02-08 23:54:14,911 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675900449564 2023-02-08 23:54:14,912 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675900449564 2023-02-08 23:54:14,912 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675900449765 2023-02-08 23:54:14,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271. 2023-02-08 23:54:14,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271. 2023-02-08 23:54:14,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32. 2023-02-08 23:54:14,912 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675900449765 2023-02-08 23:54:14,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fe03d2fe0b831458763c25fce0549d32, NAME => 'testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.', STARTKEY => '', ENDKEY => '41'} 2023-02-08 23:54:14,913 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675900449977 2023-02-08 23:54:14,913 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=335bec642c9991062859430c97058271, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:14,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:14,913 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900454913"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900454913"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900454913"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900454913"}]},"ts":"1675900454913"} 2023-02-08 23:54:14,913 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675900449977 2023-02-08 23:54:14,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR fe03d2fe0b831458763c25fce0549d32 2023-02-08 23:54:14,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:14,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for fe03d2fe0b831458763c25fce0549d32 2023-02-08 23:54:14,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for fe03d2fe0b831458763c25fce0549d32 2023-02-08 23:54:14,914 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675900450179 2023-02-08 23:54:14,914 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675900450179 2023-02-08 23:54:14,914 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675900450380 2023-02-08 23:54:14,914 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675900450380 2023-02-08 23:54:14,915 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675900450581 2023-02-08 23:54:14,915 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675900450581 2023-02-08 23:54:14,915 INFO [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fe03d2fe0b831458763c25fce0549d32 2023-02-08 23:54:14,916 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675900450775 2023-02-08 23:54:14,916 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675900450775 2023-02-08 23:54:14,918 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1 2023-02-08 23:54:14,918 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1 2023-02-08 23:54:14,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-08 23:54:14,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 335bec642c9991062859430c97058271, server=jenkins-hbase4.apache.org,42343,1675900441151 in 274 msec 2023-02-08 23:54:14,919 INFO [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fe03d2fe0b831458763c25fce0549d32 columnFamilyName cf1 2023-02-08 23:54:14,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=335bec642c9991062859430c97058271, ASSIGN in 437 msec 2023-02-08 23:54:14,934 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] throttle.PressureAwareThroughputController(145): 335bec642c9991062859430c97058271#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:14,941 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] throttle.PressureAwareThroughputController(145): 335bec642c9991062859430c97058271#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:14,945 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9-bottom 2023-02-08 23:54:14,949 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0-bottom 2023-02-08 23:54:14,958 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507-bottom 2023-02-08 23:54:14,964 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101-bottom 2023-02-08 23:54:14,967 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/.tmp/cf2/7fcddcec011348dba61c311e73900d25 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/7fcddcec011348dba61c311e73900d25 2023-02-08 23:54:14,968 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/.tmp/cf1/81cc5886885245e6ade47c8f02039a38 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/81cc5886885245e6ade47c8f02039a38 2023-02-08 23:54:14,969 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326-bottom 2023-02-08 23:54:14,975 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 335bec642c9991062859430c97058271/cf2 of 335bec642c9991062859430c97058271 into 7fcddcec011348dba61c311e73900d25(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:14,975 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559-bottom 2023-02-08 23:54:14,975 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 335bec642c9991062859430c97058271/cf1 of 335bec642c9991062859430c97058271 into 81cc5886885245e6ade47c8f02039a38(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:14,975 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 335bec642c9991062859430c97058271: 2023-02-08 23:54:14,975 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 335bec642c9991062859430c97058271: 2023-02-08 23:54:14,975 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271., storeName=335bec642c9991062859430c97058271/cf2, priority=9, startTime=1675900454909; duration=0sec 2023-02-08 23:54:14,976 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271., storeName=335bec642c9991062859430c97058271/cf1, priority=9, startTime=1675900454909; duration=0sec 2023-02-08 23:54:14,976 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:14,976 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:14,979 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c-bottom 2023-02-08 23:54:14,979 INFO [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(310): Store=fe03d2fe0b831458763c25fce0549d32/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:14,979 INFO [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fe03d2fe0b831458763c25fce0549d32 2023-02-08 23:54:14,980 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2 2023-02-08 23:54:14,980 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2 2023-02-08 23:54:14,981 INFO [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fe03d2fe0b831458763c25fce0549d32 columnFamilyName cf2 2023-02-08 23:54:14,995 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f-bottom 2023-02-08 23:54:14,999 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be-bottom 2023-02-08 23:54:15,003 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576-bottom 2023-02-08 23:54:15,007 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849-bottom 2023-02-08 23:54:15,011 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a-bottom 2023-02-08 23:54:15,015 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4-bottom 2023-02-08 23:54:15,019 DEBUG [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05-bottom 2023-02-08 23:54:15,020 INFO [StoreOpener-fe03d2fe0b831458763c25fce0549d32-1] regionserver.HStore(310): Store=fe03d2fe0b831458763c25fce0549d32/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:15,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32 2023-02-08 23:54:15,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32 2023-02-08 23:54:15,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:15,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for fe03d2fe0b831458763c25fce0549d32 2023-02-08 23:54:15,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened fe03d2fe0b831458763c25fce0549d32; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69735217, jitterRate=0.039135709404945374}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:15,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for fe03d2fe0b831458763c25fce0549d32: Running coprocessor pre-open hook at 1675900454913Writing region info on filesystem at 1675900454913Initializing all the Stores at 1675900454915 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900454915Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900454915Cleaning up temporary data from old regions at 1675900455022 (+107 ms)Cleaning up detritus from prior splits at 1675900455023 (+1 ms)Running coprocessor post-open hooks at 1675900455025 (+2 ms)Region opened successfully at 1675900455025 2023-02-08 23:54:15,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32., pid=34, masterSystemTime=1675900454794 2023-02-08 23:54:15,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:15,027 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:54:15,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:54:15,028 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:54:15,028 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32. 2023-02-08 23:54:15,028 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1912): fe03d2fe0b831458763c25fce0549d32/cf1 is initiating minor compaction (all files) 2023-02-08 23:54:15,028 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32. 2023-02-08 23:54:15,028 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1912): fe03d2fe0b831458763c25fce0549d32/cf2 is initiating minor compaction (all files) 2023-02-08 23:54:15,028 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of fe03d2fe0b831458763c25fce0549d32/cf1 in testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32. 2023-02-08 23:54:15,028 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2258): Starting compaction of fe03d2fe0b831458763c25fce0549d32/cf2 in testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32. 2023-02-08 23:54:15,028 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9-bottom] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/.tmp, totalSize=34.9 K 2023-02-08 23:54:15,028 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be-bottom] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/.tmp, totalSize=34.9 K 2023-02-08 23:54:15,029 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900449564 2023-02-08 23:54:15,029 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900449564 2023-02-08 23:54:15,029 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900449765 2023-02-08 23:54:15,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32. 2023-02-08 23:54:15,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32. 2023-02-08 23:54:15,030 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900449977 2023-02-08 23:54:15,030 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900449765 2023-02-08 23:54:15,031 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=fe03d2fe0b831458763c25fce0549d32, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:15,031 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900455030"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900455030"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900455030"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900455030"}]},"ts":"1675900455030"} 2023-02-08 23:54:15,031 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900449977 2023-02-08 23:54:15,031 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900450179 2023-02-08 23:54:15,032 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900450179 2023-02-08 23:54:15,032 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900450380 2023-02-08 23:54:15,032 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900450380 2023-02-08 23:54:15,033 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900450581 2023-02-08 23:54:15,033 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900450581 2023-02-08 23:54:15,033 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675900450775 2023-02-08 23:54:15,033 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675900450775 2023-02-08 23:54:15,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-08 23:54:15,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure fe03d2fe0b831458763c25fce0549d32, server=jenkins-hbase4.apache.org,42343,1675900441151 in 384 msec 2023-02-08 23:54:15,039 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-08 23:54:15,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fe03d2fe0b831458763c25fce0549d32, ASSIGN in 554 msec 2023-02-08 23:54:15,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 in 4.0020 sec 2023-02-08 23:54:15,050 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] throttle.PressureAwareThroughputController(145): fe03d2fe0b831458763c25fce0549d32#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:15,050 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] throttle.PressureAwareThroughputController(145): fe03d2fe0b831458763c25fce0549d32#cf1#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:15,066 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/.tmp/cf1/45d5ab3b44b444278bc6da202a03e106 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/45d5ab3b44b444278bc6da202a03e106 2023-02-08 23:54:15,066 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/.tmp/cf2/e798845715b647c1a760969887290bd4 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/e798845715b647c1a760969887290bd4 2023-02-08 23:54:15,072 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in fe03d2fe0b831458763c25fce0549d32/cf1 of fe03d2fe0b831458763c25fce0549d32 into 45d5ab3b44b444278bc6da202a03e106(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:15,072 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for fe03d2fe0b831458763c25fce0549d32: 2023-02-08 23:54:15,072 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32., storeName=fe03d2fe0b831458763c25fce0549d32/cf1, priority=9, startTime=1675900455026; duration=0sec 2023-02-08 23:54:15,072 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:15,076 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in fe03d2fe0b831458763c25fce0549d32/cf2 of fe03d2fe0b831458763c25fce0549d32 into e798845715b647c1a760969887290bd4(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:15,077 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for fe03d2fe0b831458763c25fce0549d32: 2023-02-08 23:54:15,077 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32., storeName=fe03d2fe0b831458763c25fce0549d32/cf2, priority=9, startTime=1675900455027; duration=0sec 2023-02-08 23:54:15,077 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:15,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42343] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:34096 deadline: 1675900515226, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df. is not online on jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:15,341 DEBUG [Listener at localhost/37973] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 23:54:15,349 INFO [Listener at localhost/37973] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-08 23:54:15,351 DEBUG [Listener at localhost/37973] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 23:54:15,368 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:15,368 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,368 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,370 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,370 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,370 INFO [Listener at localhost/37973] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-08 23:54:15,371 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-08 23:54:15,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:15,374 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,374 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 23:54:15,375 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900455375"}]},"ts":"1675900455375"} 2023-02-08 23:54:15,377 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-08 23:54:15,379 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-08 23:54:15,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fe03d2fe0b831458763c25fce0549d32, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=335bec642c9991062859430c97058271, UNASSIGN}] 2023-02-08 23:54:15,380 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=335bec642c9991062859430c97058271, UNASSIGN 2023-02-08 23:54:15,381 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=335bec642c9991062859430c97058271, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:15,381 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900455381"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900455381"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900455381"}]},"ts":"1675900455381"} 2023-02-08 23:54:15,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 335bec642c9991062859430c97058271, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:54:15,385 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fe03d2fe0b831458763c25fce0549d32, UNASSIGN 2023-02-08 23:54:15,386 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=fe03d2fe0b831458763c25fce0549d32, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:15,386 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900455385"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900455385"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900455385"}]},"ts":"1675900455385"} 2023-02-08 23:54:15,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure fe03d2fe0b831458763c25fce0549d32, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:54:15,475 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,476 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 23:54:15,535 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 335bec642c9991062859430c97058271 2023-02-08 23:54:15,535 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 335bec642c9991062859430c97058271, disabling compactions & flushes 2023-02-08 23:54:15,535 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271. 2023-02-08 23:54:15,535 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271. 2023-02-08 23:54:15,535 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271. after waiting 0 ms 2023-02-08 23:54:15,535 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271. 2023-02-08 23:54:15,546 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9-top] to archive 2023-02-08 23:54:15,547 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:15,550 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,551 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,552 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,554 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,555 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,556 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,558 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,560 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be-top] to archive 2023-02-08 23:54:15,561 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:15,563 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,564 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,565 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,567 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,568 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,569 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,570 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 23:54:15,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:15,574 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271. 2023-02-08 23:54:15,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 335bec642c9991062859430c97058271: Waiting for close lock at 1675900455535Running coprocessor pre-close hooks at 1675900455535Disabling compacts and flushes for region at 1675900455535Disabling writes for close at 1675900455535Writing region close event to WAL at 1675900455571 (+36 ms)Running coprocessor post-close hooks at 1675900455574 (+3 ms)Closed at 1675900455574 2023-02-08 23:54:15,576 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 335bec642c9991062859430c97058271 2023-02-08 23:54:15,576 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close fe03d2fe0b831458763c25fce0549d32 2023-02-08 23:54:15,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing fe03d2fe0b831458763c25fce0549d32, disabling compactions & flushes 2023-02-08 23:54:15,576 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32. 2023-02-08 23:54:15,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32. 2023-02-08 23:54:15,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32. after waiting 0 ms 2023-02-08 23:54:15,577 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32. 2023-02-08 23:54:15,577 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=335bec642c9991062859430c97058271, regionState=CLOSED 2023-02-08 23:54:15,577 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900455577"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900455577"}]},"ts":"1675900455577"} 2023-02-08 23:54:15,578 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9-bottom] to archive 2023-02-08 23:54:15,579 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:15,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-08 23:54:15,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 335bec642c9991062859430c97058271, server=jenkins-hbase4.apache.org,42343,1675900441151 in 195 msec 2023-02-08 23:54:15,582 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,583 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=335bec642c9991062859430c97058271, UNASSIGN in 201 msec 2023-02-08 23:54:15,584 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,586 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,587 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,589 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,590 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,591 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,594 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be-bottom] to archive 2023-02-08 23:54:15,595 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:15,596 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,598 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,599 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,601 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,602 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,604 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,605 DEBUG [StoreCloser-testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 23:54:15,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:15,611 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32. 2023-02-08 23:54:15,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for fe03d2fe0b831458763c25fce0549d32: Waiting for close lock at 1675900455576Running coprocessor pre-close hooks at 1675900455576Disabling compacts and flushes for region at 1675900455576Disabling writes for close at 1675900455577 (+1 ms)Writing region close event to WAL at 1675900455607 (+30 ms)Running coprocessor post-close hooks at 1675900455611 (+4 ms)Closed at 1675900455611 2023-02-08 23:54:15,614 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed fe03d2fe0b831458763c25fce0549d32 2023-02-08 23:54:15,614 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=fe03d2fe0b831458763c25fce0549d32, regionState=CLOSED 2023-02-08 23:54:15,614 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900455614"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900455614"}]},"ts":"1675900455614"} 2023-02-08 23:54:15,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-08 23:54:15,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure fe03d2fe0b831458763c25fce0549d32, server=jenkins-hbase4.apache.org,42343,1675900441151 in 229 msec 2023-02-08 23:54:15,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-08 23:54:15,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fe03d2fe0b831458763c25fce0549d32, UNASSIGN in 238 msec 2023-02-08 23:54:15,622 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900455622"}]},"ts":"1675900455622"} 2023-02-08 23:54:15,623 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-08 23:54:15,625 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-08 23:54:15,627 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 255 msec 2023-02-08 23:54:15,677 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,677 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 23:54:15,678 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-08 23:54:15,679 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,679 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,680 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-08 23:54:15,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:15,682 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:15,683 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,683 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,683 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:15,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-08 23:54:15,690 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,690 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271 2023-02-08 23:54:15,690 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32 2023-02-08 23:54:15,692 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/recovered.edits] 2023-02-08 23:54:15,692 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/recovered.edits] 2023-02-08 23:54:15,693 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/recovered.edits] 2023-02-08 23:54:15,696 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/45d5ab3b44b444278bc6da202a03e106 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf1/45d5ab3b44b444278bc6da202a03e106 2023-02-08 23:54:15,697 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/.regioninfo to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/.regioninfo 2023-02-08 23:54:15,697 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/05076176f46d4ccfafd9aaf5c9524aa9 2023-02-08 23:54:15,699 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/26db8435edd1481db01a2586f88d3bc0 2023-02-08 23:54:15,699 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/e798845715b647c1a760969887290bd4 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/cf2/e798845715b647c1a760969887290bd4 2023-02-08 23:54:15,701 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,701 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/331696e6c7f44124ab8dfc6480ff0507 2023-02-08 23:54:15,702 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,702 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3aa70ba641eb4587b40ee0c0e7892101 2023-02-08 23:54:15,703 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/.regioninfo to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/.regioninfo 2023-02-08 23:54:15,704 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/3c19d9bcc3534afd87e2766e56865326 2023-02-08 23:54:15,704 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,706 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,706 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/6a77b2b4cce440d89ecab656c0e50559 2023-02-08 23:54:15,707 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/05076176f46d4ccfafd9aaf5c9524aa9.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,708 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf1/86a819126854495c8db28c148ce0456c 2023-02-08 23:54:15,712 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,713 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/26db8435edd1481db01a2586f88d3bc0.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,714 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,715 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/331696e6c7f44124ab8dfc6480ff0507.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,716 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/17ce6af538d441f4aef2dc6d6dacff6f 2023-02-08 23:54:15,716 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,717 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/3aa70ba641eb4587b40ee0c0e7892101.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,717 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/932d721e58884dd3b2b8823bf2ff21be 2023-02-08 23:54:15,718 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/3c19d9bcc3534afd87e2766e56865326.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,720 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/a723a1914a244cd880e78894b22ee576 2023-02-08 23:54:15,721 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/6a77b2b4cce440d89ecab656c0e50559.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,722 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,722 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/b69b10c7ba0542be8bd09b0826df7849 2023-02-08 23:54:15,723 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf1/86a819126854495c8db28c148ce0456c.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,724 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/cd4287af543c4f2091447436b58f328a 2023-02-08 23:54:15,724 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,725 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4 2023-02-08 23:54:15,725 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,727 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,727 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/17ce6af538d441f4aef2dc6d6dacff6f.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,727 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/cf2/fe147b97248a4ca4b8420ea4832f9c05 2023-02-08 23:54:15,729 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/932d721e58884dd3b2b8823bf2ff21be.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,729 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,731 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/a723a1914a244cd880e78894b22ee576.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,731 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,731 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/recovered.edits/85.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df/recovered.edits/85.seqid 2023-02-08 23:54:15,732 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,732 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/b69b10c7ba0542be8bd09b0826df7849.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,733 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/335bec642c9991062859430c97058271/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,734 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/cd4287af543c4f2091447436b58f328a.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,735 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/d1b0e4de35554c73a5b4ab0b7d37dce4.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,736 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/81cc5886885245e6ade47c8f02039a38 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf1/81cc5886885245e6ade47c8f02039a38 2023-02-08 23:54:15,737 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/fe03d2fe0b831458763c25fce0549d32/cf2/fe147b97248a4ca4b8420ea4832f9c05.af07d15277922930e96dbc9b857cb0df 2023-02-08 23:54:15,739 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/7fcddcec011348dba61c311e73900d25 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/cf2/7fcddcec011348dba61c311e73900d25 2023-02-08 23:54:15,740 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/recovered.edits/90.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32/recovered.edits/90.seqid 2023-02-08 23:54:15,741 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/fe03d2fe0b831458763c25fce0549d32 2023-02-08 23:54:15,742 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/recovered.edits/90.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271/recovered.edits/90.seqid 2023-02-08 23:54:15,743 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitWithoutPONR/335bec642c9991062859430c97058271 2023-02-08 23:54:15,743 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-08 23:54:15,745 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:15,748 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-08 23:54:15,751 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-08 23:54:15,752 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:15,752 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-08 23:54:15,753 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900455752"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:15,753 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900455752"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:15,753 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900455752"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:15,755 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 23:54:15,755 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => af07d15277922930e96dbc9b857cb0df, NAME => 'testSplitWithoutPONR,,1675900449055.af07d15277922930e96dbc9b857cb0df.', STARTKEY => '', ENDKEY => ''}, {ENCODED => fe03d2fe0b831458763c25fce0549d32, NAME => 'testSplitWithoutPONR,,1675900451038.fe03d2fe0b831458763c25fce0549d32.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 335bec642c9991062859430c97058271, NAME => 'testSplitWithoutPONR,41,1675900451038.335bec642c9991062859430c97058271.', STARTKEY => '41', ENDKEY => ''}] 2023-02-08 23:54:15,755 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-08 23:54:15,755 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675900455755"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:15,757 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-08 23:54:15,758 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:15,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 79 msec 2023-02-08 23:54:15,784 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,784 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-08 23:54:15,785 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-08 23:54:15,802 INFO [Listener at localhost/37973] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=400 (was 390) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:42954 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:43008 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:42968 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:51426 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:42343-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:43020 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:39338 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:42916 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:58170 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:42992 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49c176f2-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:58436 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:42982 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:58448 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:58174 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:58422 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:42938 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49c176f2-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-955025252_15 at /127.0.0.1:51318 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=855 (was 853) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=264 (was 287), ProcessCount=168 (was 168), AvailableMemoryMB=3384 (was 3621) 2023-02-08 23:54:15,816 INFO [Listener at localhost/37973] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=400, OpenFileDescriptor=855, MaxFileDescriptor=60000, SystemLoadAverage=264, ProcessCount=168, AvailableMemoryMB=3383 2023-02-08 23:54:15,817 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:15,817 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,817 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,820 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 23:54:15,821 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 23:54:15,822 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:15,824 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:54:15,825 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:54:15,827 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:15,827 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596 empty. 2023-02-08 23:54:15,828 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:15,828 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-08 23:54:15,839 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:15,840 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8ef507e9697a1fef706cfdf1625d1596, NAME => 'testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp 2023-02-08 23:54:15,849 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:15,849 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 8ef507e9697a1fef706cfdf1625d1596, disabling compactions & flushes 2023-02-08 23:54:15,849 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:15,849 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:15,849 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. after waiting 0 ms 2023-02-08 23:54:15,850 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:15,850 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:15,850 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 8ef507e9697a1fef706cfdf1625d1596: Waiting for close lock at 1675900455849Disabling compacts and flushes for region at 1675900455849Disabling writes for close at 1675900455850 (+1 ms)Writing region close event to WAL at 1675900455850Closed at 1675900455850 2023-02-08 23:54:15,852 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:54:15,853 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900455853"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900455853"}]},"ts":"1675900455853"} 2023-02-08 23:54:15,855 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:54:15,856 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:54:15,856 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900455856"}]},"ts":"1675900455856"} 2023-02-08 23:54:15,858 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-08 23:54:15,861 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:54:15,862 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:54:15,862 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:54:15,862 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:54:15,862 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 23:54:15,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN}] 2023-02-08 23:54:15,864 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:15,865 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=false 2023-02-08 23:54:16,015 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:54:16,017 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=8ef507e9697a1fef706cfdf1625d1596, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,017 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900456016"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900456016"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900456016"}]},"ts":"1675900456016"} 2023-02-08 23:54:16,019 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:16,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:16,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8ef507e9697a1fef706cfdf1625d1596, NAME => 'testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:16,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:16,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:16,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:16,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:16,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:16,176 INFO [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:16,178 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1 2023-02-08 23:54:16,178 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1 2023-02-08 23:54:16,178 INFO [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8ef507e9697a1fef706cfdf1625d1596 columnFamilyName cf1 2023-02-08 23:54:16,179 INFO [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(310): Store=8ef507e9697a1fef706cfdf1625d1596/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:16,179 INFO [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:16,181 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2 2023-02-08 23:54:16,181 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2 2023-02-08 23:54:16,181 INFO [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8ef507e9697a1fef706cfdf1625d1596 columnFamilyName cf2 2023-02-08 23:54:16,181 INFO [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(310): Store=8ef507e9697a1fef706cfdf1625d1596/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:16,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:16,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:16,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:16,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:16,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:16,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8ef507e9697a1fef706cfdf1625d1596; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71215473, jitterRate=0.06119324266910553}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:16,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8ef507e9697a1fef706cfdf1625d1596: Running coprocessor pre-open hook at 1675900456175Writing region info on filesystem at 1675900456175Initializing all the Stores at 1675900456176 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900456176Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900456176Cleaning up temporary data from old regions at 1675900456183 (+7 ms)Cleaning up detritus from prior splits at 1675900456184 (+1 ms)Running coprocessor post-open hooks at 1675900456188 (+4 ms)Region opened successfully at 1675900456188 2023-02-08 23:54:16,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596., pid=43, masterSystemTime=1675900456171 2023-02-08 23:54:16,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:16,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:16,191 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=8ef507e9697a1fef706cfdf1625d1596, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,191 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900456191"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900456191"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900456191"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900456191"}]},"ts":"1675900456191"} 2023-02-08 23:54:16,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-08 23:54:16,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 in 174 msec 2023-02-08 23:54:16,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-08 23:54:16,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN in 332 msec 2023-02-08 23:54:16,197 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:54:16,198 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900456197"}]},"ts":"1675900456197"} 2023-02-08 23:54:16,199 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-08 23:54:16,205 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:54:16,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 384 msec 2023-02-08 23:54:16,332 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:16,332 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:16,333 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:16,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 23:54:16,335 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 23:54:16,335 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,335 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 23:54:16,335 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 23:54:16,335 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,335 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,337 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,337 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,337 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,337 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,337 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,337 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:16,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,338 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:16,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:16,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:16,339 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:16,339 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:16,339 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. started... 2023-02-08 23:54:16,341 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8ef507e9697a1fef706cfdf1625d1596 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 23:54:16,342 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,342 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,342 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,342 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:16,342 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,342 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:16,362 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/ee1bdb24106a42ccba67cd3dc436cf7c 2023-02-08 23:54:16,376 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/a97350aa847344dfadd01fab3bff36a0 2023-02-08 23:54:16,381 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/ee1bdb24106a42ccba67cd3dc436cf7c as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ee1bdb24106a42ccba67cd3dc436cf7c 2023-02-08 23:54:16,385 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ee1bdb24106a42ccba67cd3dc436cf7c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:54:16,386 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/a97350aa847344dfadd01fab3bff36a0 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/a97350aa847344dfadd01fab3bff36a0 2023-02-08 23:54:16,391 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/a97350aa847344dfadd01fab3bff36a0, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:54:16,392 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 8ef507e9697a1fef706cfdf1625d1596 in 52ms, sequenceid=6, compaction requested=false 2023-02-08 23:54:16,392 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-08 23:54:16,393 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8ef507e9697a1fef706cfdf1625d1596: 2023-02-08 23:54:16,393 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:16,393 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:16,393 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:16,393 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,393 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:16,393 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:16,398 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,398 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:16,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:16,398 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,399 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:16,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:16,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:16,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:16,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 23:54:16,401 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 23:54:16,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28fe9aa[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:16,401 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,402 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,402 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,402 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,402 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:16,402 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,402 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:16,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,402 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:16,403 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:16,402 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:16,403 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:16,403 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:16,403 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:16,403 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:16,403 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:16,404 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:16,404 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,404 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:16,405 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:16,405 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:16,405 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,405 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:16,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:16,405 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,405 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:16,405 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:16,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:16,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:16,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:16,408 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 23:54:16,408 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 23:54:16,409 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:16,409 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:16,409 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 23:54:16,409 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:16,412 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,412 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:16,412 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,412 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,412 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,412 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,412 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,412 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,413 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,414 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:16,416 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,416 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,417 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,419 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,419 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,419 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,420 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,420 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,420 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,420 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,419 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,420 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 23:54:16,420 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:16,420 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,420 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,420 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,421 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:16,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,421 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:16,421 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,421 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,421 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,421 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,421 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,421 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,521 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:16,522 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:16,522 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:16,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:16,539 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:16,539 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:16,540 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:16,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:16,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 23:54:16,541 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 23:54:16,541 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,541 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 23:54:16,541 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 23:54:16,542 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,542 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,543 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,543 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,543 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,543 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,543 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,544 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:16,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,545 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:16,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,545 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,545 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:16,545 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:16,545 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:16,545 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:16,545 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:16,545 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,545 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:16,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,546 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:16,545 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:16,546 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:16,546 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:16,546 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,546 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:16,546 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:16,546 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:16,546 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:16,546 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:16,546 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:16,546 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:16,546 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,546 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:16,546 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:16,546 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:16,547 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:16,547 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. started... 2023-02-08 23:54:16,547 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8ef507e9697a1fef706cfdf1625d1596 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:16,547 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:16,548 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:16,560 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/bd061cdf305c4b5e852649a15c957abd 2023-02-08 23:54:16,578 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/6e9a0b80488d4d17b6100d56d891937e 2023-02-08 23:54:16,583 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/bd061cdf305c4b5e852649a15c957abd as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/bd061cdf305c4b5e852649a15c957abd 2023-02-08 23:54:16,588 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/bd061cdf305c4b5e852649a15c957abd, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:54:16,589 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/6e9a0b80488d4d17b6100d56d891937e as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/6e9a0b80488d4d17b6100d56d891937e 2023-02-08 23:54:16,593 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/6e9a0b80488d4d17b6100d56d891937e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:54:16,594 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8ef507e9697a1fef706cfdf1625d1596 in 47ms, sequenceid=19, compaction requested=false 2023-02-08 23:54:16,595 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8ef507e9697a1fef706cfdf1625d1596: 2023-02-08 23:54:16,595 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:16,595 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:16,595 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:16,595 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,595 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:16,595 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:16,601 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,601 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:16,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:16,601 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,601 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:16,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:16,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:16,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:16,603 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 23:54:16,603 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5086f630[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:16,603 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 23:54:16,603 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,604 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,604 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,605 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,605 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:16,605 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:16,605 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:16,605 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:16,605 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,605 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:16,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,605 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:16,605 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:16,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,605 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:16,605 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:16,605 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:16,608 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,608 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:16,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:16,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:16,608 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:16,609 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,609 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,609 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:16,609 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:16,609 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:16,609 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:16,609 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:16,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:16,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:16,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:16,612 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 23:54:16,612 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 23:54:16,612 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:16,613 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:16,613 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 23:54:16,613 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:16,614 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,614 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,614 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,614 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,614 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:16,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:16,615 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,615 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,614 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:16,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,615 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,616 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:16,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,618 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:16,618 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,618 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,623 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,623 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 23:54:16,623 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:16,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,624 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:16,623 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,624 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,624 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,624 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,624 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,624 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,724 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:16,724 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:16,724 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:16,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:16,739 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:16,739 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:16,740 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:16,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:16,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 23:54:16,742 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 23:54:16,742 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,742 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 23:54:16,742 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 23:54:16,742 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,742 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,743 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,743 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,743 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,743 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,744 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,744 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:16,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,745 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:16,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,745 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:16,745 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:16,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:16,746 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:16,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:16,746 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:16,746 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:16,746 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:16,747 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,748 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,748 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,748 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:16,748 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,748 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:16,751 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:16,751 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:16,751 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:16,752 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. started... 2023-02-08 23:54:16,752 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8ef507e9697a1fef706cfdf1625d1596 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:16,767 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/ecebd88d78864396909427b65832bf09 2023-02-08 23:54:16,796 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/e6577c5f177642aeabb0233d6b5e0f82 2023-02-08 23:54:16,801 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/ecebd88d78864396909427b65832bf09 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ecebd88d78864396909427b65832bf09 2023-02-08 23:54:16,805 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ecebd88d78864396909427b65832bf09, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:54:16,806 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/e6577c5f177642aeabb0233d6b5e0f82 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e6577c5f177642aeabb0233d6b5e0f82 2023-02-08 23:54:16,812 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e6577c5f177642aeabb0233d6b5e0f82, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:54:16,813 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8ef507e9697a1fef706cfdf1625d1596 in 61ms, sequenceid=32, compaction requested=true 2023-02-08 23:54:16,813 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8ef507e9697a1fef706cfdf1625d1596: 2023-02-08 23:54:16,813 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:16,813 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:16,813 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:16,813 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,813 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:16,813 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:16,815 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,815 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:16,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:16,815 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,815 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:16,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:16,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:16,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:16,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 23:54:16,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41ee6381[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:16,817 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 23:54:16,817 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,818 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,818 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,818 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:16,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:16,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:16,818 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,819 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:16,819 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:16,819 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:16,821 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,821 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:16,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:16,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:16,821 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,821 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:16,821 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:16,821 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,821 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:16,822 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:16,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:16,822 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,822 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:16,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:16,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:16,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:16,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 23:54:16,825 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 23:54:16,825 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:16,825 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:16,825 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 23:54:16,825 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:16,826 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,826 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,826 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,826 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,827 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,826 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,826 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,827 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:16,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,827 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,828 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:16,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,829 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:16,829 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:16,830 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,835 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,835 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,835 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,835 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,835 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,835 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 23:54:16,835 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,835 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,835 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,835 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,835 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,835 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:16,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,835 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,835 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,836 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:16,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,836 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,836 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,836 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,836 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:16,836 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:16,836 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,836 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,936 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:16,936 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:16,936 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:16,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:16,952 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:16,952 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:16,953 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:16,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:16,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 23:54:16,955 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 23:54:16,955 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,955 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 23:54:16,955 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 23:54:16,955 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,955 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,956 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,956 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,956 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,956 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,957 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:16,957 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,957 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,957 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,957 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:16,958 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:16,958 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:16,958 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:16,958 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:16,958 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:16,958 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:16,958 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:16,958 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,958 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:16,958 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:16,958 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:16,958 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,958 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:16,959 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:16,959 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:16,959 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. started... 2023-02-08 23:54:16,959 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8ef507e9697a1fef706cfdf1625d1596 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:16,960 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,962 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,962 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:16,962 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,962 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:16,963 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:16,975 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/11c46ab231134ad0bc579fbfe94b2f75 2023-02-08 23:54:16,987 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/f693ef97d443479bbe8348015751e36e 2023-02-08 23:54:16,991 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/11c46ab231134ad0bc579fbfe94b2f75 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/11c46ab231134ad0bc579fbfe94b2f75 2023-02-08 23:54:16,996 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/11c46ab231134ad0bc579fbfe94b2f75, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:54:16,996 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/f693ef97d443479bbe8348015751e36e as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/f693ef97d443479bbe8348015751e36e 2023-02-08 23:54:17,001 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/f693ef97d443479bbe8348015751e36e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:54:17,002 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8ef507e9697a1fef706cfdf1625d1596 in 43ms, sequenceid=45, compaction requested=true 2023-02-08 23:54:17,002 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8ef507e9697a1fef706cfdf1625d1596: 2023-02-08 23:54:17,002 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:17,002 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:17,002 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:17,002 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,002 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:17,002 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:17,004 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,004 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:17,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:17,005 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,005 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:17,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:17,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:17,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:17,007 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 23:54:17,007 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76baf493[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:17,007 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 23:54:17,007 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,008 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,008 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,008 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,008 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:17,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,008 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:17,009 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:17,009 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:17,008 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,009 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:17,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,009 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:17,009 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:17,009 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:17,009 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:17,009 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:17,010 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:17,010 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:17,010 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,010 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,010 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:17,010 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:17,014 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:17,014 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,014 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:17,015 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:17,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:17,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:17,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:17,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:17,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:17,017 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 23:54:17,017 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 23:54:17,018 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:17,018 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:17,018 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 23:54:17,018 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:17,019 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,019 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,019 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,019 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,019 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,019 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:17,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,019 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,020 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:17,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,020 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:17,021 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:17,022 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,022 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,023 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,025 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,025 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,025 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,025 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,026 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 23:54:17,026 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,026 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:17,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,026 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,025 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,025 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,025 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,026 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,026 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,026 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,026 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:17,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,026 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,026 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:17,027 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,027 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,027 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,027 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,027 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,127 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:17,127 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:17,127 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:17,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:17,143 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:17,143 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:17,143 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:17,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:17,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 23:54:17,145 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 23:54:17,145 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:17,145 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 23:54:17,145 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 23:54:17,146 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,146 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,147 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,147 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,147 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,147 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,148 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,148 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:17,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:17,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:17,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,149 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:17,149 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:17,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:17,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,149 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:17,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:17,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:17,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:17,149 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:17,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:17,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:17,149 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:17,149 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:17,149 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:17,149 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:17,150 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:17,150 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:17,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:17,150 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:17,150 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:17,150 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:17,150 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:17,150 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:17,150 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:17,150 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:17,150 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:17,150 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:17,150 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. started... 2023-02-08 23:54:17,150 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8ef507e9697a1fef706cfdf1625d1596 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:17,158 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,158 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,158 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,159 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:17,159 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,159 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:17,165 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/33258c4a961642df92d60ef8669f6c9a 2023-02-08 23:54:17,178 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/94dcf9c7e766468cacc6174fc3e6f340 2023-02-08 23:54:17,183 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/33258c4a961642df92d60ef8669f6c9a as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/33258c4a961642df92d60ef8669f6c9a 2023-02-08 23:54:17,187 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/33258c4a961642df92d60ef8669f6c9a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:54:17,188 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/94dcf9c7e766468cacc6174fc3e6f340 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/94dcf9c7e766468cacc6174fc3e6f340 2023-02-08 23:54:17,191 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/94dcf9c7e766468cacc6174fc3e6f340, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:54:17,192 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8ef507e9697a1fef706cfdf1625d1596 in 42ms, sequenceid=58, compaction requested=true 2023-02-08 23:54:17,192 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8ef507e9697a1fef706cfdf1625d1596: 2023-02-08 23:54:17,192 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:17,192 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:17,194 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,194 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:17,195 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,195 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:17,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 23:54:17,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b859157[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:17,197 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 23:54:17,197 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,198 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,198 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,198 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,198 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:17,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,198 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:17,198 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:17,198 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:17,198 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:17,198 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,199 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:17,199 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:17,199 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:17,198 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:17,199 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:17,204 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,204 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:17,204 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,204 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:17,204 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:17,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:17,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:17,204 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,204 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:17,205 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:17,205 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:17,205 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:17,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:17,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:17,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:17,207 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 23:54:17,207 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 23:54:17,207 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:17,207 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:17,207 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 23:54:17,207 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:17,209 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,209 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,209 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,209 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,209 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,209 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:17,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,209 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,209 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,210 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,211 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:17,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,212 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,212 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:17,213 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,216 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,216 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,216 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,216 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 23:54:17,216 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,216 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,216 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,217 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,216 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:17,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,216 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,217 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,217 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,217 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:17,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,217 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,217 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,217 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:17,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,217 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,218 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,218 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,218 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,218 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,218 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,318 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:17,318 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:17,318 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:17,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:17,340 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:17,340 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:17,342 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:17,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:17,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 23:54:17,345 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 23:54:17,345 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:17,345 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 23:54:17,345 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 23:54:17,346 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,346 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,347 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,347 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,347 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,347 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,349 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,349 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:17,349 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,349 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,349 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,350 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:17,350 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,350 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 23:54:17,350 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:17,350 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:17,350 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:17,350 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:17,350 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:17,350 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 23:54:17,350 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:17,350 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:17,350 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:17,350 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:17,350 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:17,351 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:17,351 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. started... 2023-02-08 23:54:17,351 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8ef507e9697a1fef706cfdf1625d1596 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:17,352 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,353 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,353 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,353 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:17,353 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,353 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:17,369 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/1d460adc251c414d8f9e46ab583e599b 2023-02-08 23:54:17,382 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/eccab892ccee400a9059552ab196e5d8 2023-02-08 23:54:17,387 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/1d460adc251c414d8f9e46ab583e599b as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/1d460adc251c414d8f9e46ab583e599b 2023-02-08 23:54:17,392 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/1d460adc251c414d8f9e46ab583e599b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:54:17,392 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/eccab892ccee400a9059552ab196e5d8 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/eccab892ccee400a9059552ab196e5d8 2023-02-08 23:54:17,397 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/eccab892ccee400a9059552ab196e5d8, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:54:17,398 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8ef507e9697a1fef706cfdf1625d1596 in 47ms, sequenceid=71, compaction requested=true 2023-02-08 23:54:17,398 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8ef507e9697a1fef706cfdf1625d1596: 2023-02-08 23:54:17,398 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:17,398 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:17,398 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:17,398 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,398 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 23:54:17,398 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 23:54:17,400 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,400 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:17,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:17,400 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,400 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:17,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:17,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:17,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:17,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 23:54:17,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c55d161[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:17,402 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 23:54:17,402 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,403 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,403 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,403 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,403 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:17,404 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:17,404 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:17,404 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,404 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:17,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,404 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:17,404 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:54:17,404 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:17,404 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:17,404 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 23:54:17,404 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:17,405 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:17,405 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,405 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:17,407 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:17,407 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,407 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,407 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:17,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:17,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:17,408 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 23:54:17,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:17,408 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,408 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 23:54:17,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:17,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:17,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:17,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 23:54:17,411 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 23:54:17,411 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:17,411 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:17,411 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 23:54:17,411 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:17,413 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,413 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,413 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,413 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,413 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,413 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,413 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,413 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,414 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,414 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:17,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,416 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,416 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:17,416 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 23:54:17,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,422 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 23:54:17,423 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:17,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,423 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,423 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,423 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:17,422 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,423 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:17,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,423 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 23:54:17,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,423 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,424 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,424 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:17,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,424 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:17,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,424 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 23:54:17,424 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 23:54:17,523 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:17,524 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:17,524 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:17,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:17,785 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-08 23:54:17,786 DEBUG [Listener at localhost/37973] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:17,787 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:17,788 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:17,788 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:17,788 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:17,789 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:17,789 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:17,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:17,789 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:18,037 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,037 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:18,037 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:18,038 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:18,038 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-08 23:54:18,038 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:18,038 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:18,038 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:18,038 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:18,039 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:18,040 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-08 23:54:18,040 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:18,041 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:18,041 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:18,042 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:18,042 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:18,042 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:18,043 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:18,043 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:18,043 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:18,044 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:18,044 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:18,044 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:18,044 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:18,045 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,045 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:18,045 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,045 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,045 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,045 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 23:54:18,046 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:18,046 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:18,047 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900488047 2023-02-08 23:54:18,048 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:18,048 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,051 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:18,051 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:18,051 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:18,051 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900518051 2023-02-08 23:54:18,051 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900578051 2023-02-08 23:54:18,051 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 23:54:18,051 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:18,055 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:18,055 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:18,055 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:18,301 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,302 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:18,302 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:18,302 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:18,302 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:18,302 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:18,302 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:18,302 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:18,302 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:18,303 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:18,304 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-08 23:54:18,304 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:18,305 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:18,305 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:18,305 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:18,306 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:18,306 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:18,307 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:18,307 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:18,307 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:18,308 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:18,308 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:18,308 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:18,308 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:18,308 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,309 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:18,309 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,309 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,309 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,309 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 23:54:18,309 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:18,310 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:18,313 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900488313 2023-02-08 23:54:18,315 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:18,315 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,318 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:18,318 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:18,319 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:18,319 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900518319 2023-02-08 23:54:18,319 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900578319 2023-02-08 23:54:18,319 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:18,319 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:18,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN}] 2023-02-08 23:54:18,322 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:18,322 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:18,322 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:18,569 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,569 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:18,569 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:18,570 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:18,571 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:18,571 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:18,571 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:18,571 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:18,571 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:18,572 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:18,573 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-08 23:54:18,573 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:18,574 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:18,574 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:18,574 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:18,575 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:18,575 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:18,575 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:18,575 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:18,576 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:18,576 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:18,577 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:18,577 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:18,578 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:18,578 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,578 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:18,578 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,578 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,578 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,579 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 23:54:18,579 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:18,580 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:18,580 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900488580 2023-02-08 23:54:18,580 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:18,582 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,583 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:18,584 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:18,584 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:18,584 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900518584 2023-02-08 23:54:18,584 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900578584 2023-02-08 23:54:18,584 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:18,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN}] 2023-02-08 23:54:18,584 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:18,586 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:18,587 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:18,588 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900458587"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900458587"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900458587"}]},"ts":"1675900458587"} 2023-02-08 23:54:18,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:18,589 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:18,589 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:18,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:18,589 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:18,834 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,834 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:18,834 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:18,835 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:18,837 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:18,837 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:18,837 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:18,837 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:18,837 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:18,838 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:18,839 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 23:54:18,839 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 23:54:18,840 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:18,840 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:18,840 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:18,841 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:18,841 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:18,842 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:18,842 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:18,842 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:18,843 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:18,843 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:18,843 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:18,843 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:18,844 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:18,845 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,845 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:18,845 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:18,845 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,845 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,845 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:18,845 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:18,846 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:18,846 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-08 23:54:18,846 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:18,847 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:18,847 INFO [Listener at localhost/37973] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596 to restore RIT 2023-02-08 23:54:18,851 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900488851 2023-02-08 23:54:18,851 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:18,853 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:18,855 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:18,855 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:18,855 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:18,855 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900518855 2023-02-08 23:54:18,855 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900578855 2023-02-08 23:54:18,856 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 23:54:18,856 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:18,856 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900458856"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900458856"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900458856"}]},"ts":"1675900458856"} 2023-02-08 23:54:18,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:18,859 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:19,012 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8ef507e9697a1fef706cfdf1625d1596, disabling compactions & flushes 2023-02-08 23:54:19,013 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:19,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:19,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. after waiting 0 ms 2023-02-08 23:54:19,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:19,013 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 8ef507e9697a1fef706cfdf1625d1596 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 23:54:19,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/4f382ff1b13845d0b8a7073151178f1a 2023-02-08 23:54:19,038 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/e2c4cecb086e4e8eb2711f49b4425a4c 2023-02-08 23:54:19,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/4f382ff1b13845d0b8a7073151178f1a as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/4f382ff1b13845d0b8a7073151178f1a 2023-02-08 23:54:19,046 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/4f382ff1b13845d0b8a7073151178f1a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:54:19,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/e2c4cecb086e4e8eb2711f49b4425a4c as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e2c4cecb086e4e8eb2711f49b4425a4c 2023-02-08 23:54:19,051 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e2c4cecb086e4e8eb2711f49b4425a4c, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:54:19,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 8ef507e9697a1fef706cfdf1625d1596 in 39ms, sequenceid=82, compaction requested=true 2023-02-08 23:54:19,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 23:54:19,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:19,057 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:19,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8ef507e9697a1fef706cfdf1625d1596: Waiting for close lock at 1675900459012Running coprocessor pre-close hooks at 1675900459012Disabling compacts and flushes for region at 1675900459012Disabling writes for close at 1675900459013 (+1 ms)Obtaining lock to block concurrent updates at 1675900459013Preparing flush snapshotting stores in 8ef507e9697a1fef706cfdf1625d1596 at 1675900459013Finished memstore snapshotting testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675900459013Flushing stores of testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. at 1675900459014 (+1 ms)Flushing 8ef507e9697a1fef706cfdf1625d1596/cf1: creating writer at 1675900459015 (+1 ms)Flushing 8ef507e9697a1fef706cfdf1625d1596/cf1: appending metadata at 1675900459017 (+2 ms)Flushing 8ef507e9697a1fef706cfdf1625d1596/cf1: closing flushed file at 1675900459017Flushing 8ef507e9697a1fef706cfdf1625d1596/cf2: creating writer at 1675900459029 (+12 ms)Flushing 8ef507e9697a1fef706cfdf1625d1596/cf2: appending metadata at 1675900459030 (+1 ms)Flushing 8ef507e9697a1fef706cfdf1625d1596/cf2: closing flushed file at 1675900459030Flushing 8ef507e9697a1fef706cfdf1625d1596/cf1: reopening flushed file at 1675900459043 (+13 ms)Flushing 8ef507e9697a1fef706cfdf1625d1596/cf2: reopening flushed file at 1675900459048 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 8ef507e9697a1fef706cfdf1625d1596 in 39ms, sequenceid=82, compaction requested=true at 1675900459052 (+4 ms)Writing region close event to WAL at 1675900459054 (+2 ms)Running coprocessor post-close hooks at 1675900459057 (+3 ms)Closed at 1675900459057 2023-02-08 23:54:19,059 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,060 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED 2023-02-08 23:54:19,060 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900459060"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900459060"}]},"ts":"1675900459060"} 2023-02-08 23:54:19,061 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:19,061 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:19,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:19,061 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:19,106 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,106 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:19,106 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:19,106 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:19,107 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:19,107 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:19,107 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:19,107 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:19,107 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:19,108 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:19,109 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 23:54:19,109 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-08 23:54:19,109 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-08 23:54:19,109 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:19,110 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:19,110 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:19,110 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:19,110 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:19,110 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:19,111 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:19,111 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:19,111 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:19,111 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:19,111 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:19,111 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:19,112 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:19,112 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,112 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:19,112 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:19,112 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:19,113 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,113 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,113 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,113 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:19,113 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:19,113 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:19,113 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 23:54:19,114 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:19,114 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:19,114 INFO [Listener at localhost/37973] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596 to restore RIT 2023-02-08 23:54:19,114 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900489114 2023-02-08 23:54:19,115 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:19,115 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:19,118 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:19,118 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:19,118 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:19,118 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900519118 2023-02-08 23:54:19,118 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900579118 2023-02-08 23:54:19,118 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:19,118 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:19,119 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900459118"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900459118"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900459118"}]},"ts":"1675900459118"} 2023-02-08 23:54:19,120 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:19,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-08 23:54:19,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 in 261 msec 2023-02-08 23:54:19,123 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:19,123 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:19,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:19,123 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:19,368 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,369 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:19,369 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:19,369 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:19,370 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:19,370 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:19,370 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:19,370 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:19,370 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:19,371 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:19,372 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 23:54:19,372 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-08 23:54:19,372 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:19,373 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:19,373 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:19,373 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:19,373 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:19,374 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:19,374 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:19,374 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:19,374 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:19,375 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:19,375 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:19,375 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:19,375 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:19,375 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:19,376 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,376 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:19,376 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:19,376 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:19,377 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,377 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,377 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,377 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:19,377 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:19,377 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 23:54:19,378 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:19,378 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:19,378 INFO [Listener at localhost/37973] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596 to restore RIT 2023-02-08 23:54:19,379 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900489378 2023-02-08 23:54:19,379 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:19,380 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:19,382 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:19,382 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:19,382 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:19,382 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900519382 2023-02-08 23:54:19,382 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900579382 2023-02-08 23:54:19,382 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:19,383 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:19,384 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:19,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-08 23:54:19,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN in 800 msec 2023-02-08 23:54:19,391 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:19,391 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:19,391 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:19,633 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,633 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:19,633 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:19,633 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:19,633 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:19,633 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:19,633 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:19,633 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:19,633 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:19,634 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:19,635 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 23:54:19,635 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:54:19,635 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:19,635 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:19,636 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:19,636 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:19,636 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:19,636 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:19,637 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:19,637 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:19,637 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:19,637 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:19,637 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:19,638 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:19,638 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:19,638 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:19,638 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,639 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:19,639 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:19,639 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:19,639 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:19,639 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:19,639 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 23:54:19,640 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:19,640 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:19,641 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900489641 2023-02-08 23:54:19,642 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,642 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:19,645 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:19,645 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:19,645 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:19,645 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900519645 2023-02-08 23:54:19,645 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900579645 2023-02-08 23:54:19,645 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 23:54:19,645 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,645 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,645 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,650 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:19,660 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=8ef507e9697a1fef706cfdf1625d1596, threads=14 2023-02-08 23:54:19,661 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/6e9a0b80488d4d17b6100d56d891937e for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,661 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/94dcf9c7e766468cacc6174fc3e6f340 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,661 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/a97350aa847344dfadd01fab3bff36a0 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,661 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e2c4cecb086e4e8eb2711f49b4425a4c for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,661 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e6577c5f177642aeabb0233d6b5e0f82 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,661 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/eccab892ccee400a9059552ab196e5d8 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,662 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/f693ef97d443479bbe8348015751e36e for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,662 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/11c46ab231134ad0bc579fbfe94b2f75 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,662 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/1d460adc251c414d8f9e46ab583e599b for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,662 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/33258c4a961642df92d60ef8669f6c9a for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,662 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/4f382ff1b13845d0b8a7073151178f1a for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,662 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/bd061cdf305c4b5e852649a15c957abd for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,664 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ecebd88d78864396909427b65832bf09 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,664 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ee1bdb24106a42ccba67cd3dc436cf7c for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,773 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/a97350aa847344dfadd01fab3bff36a0 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,781 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/33258c4a961642df92d60ef8669f6c9a for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,787 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/4f382ff1b13845d0b8a7073151178f1a for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,788 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e2c4cecb086e4e8eb2711f49b4425a4c for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,796 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ecebd88d78864396909427b65832bf09 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,832 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/11c46ab231134ad0bc579fbfe94b2f75 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,842 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/94dcf9c7e766468cacc6174fc3e6f340 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,844 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/bd061cdf305c4b5e852649a15c957abd for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,846 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ee1bdb24106a42ccba67cd3dc436cf7c for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,848 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/eccab892ccee400a9059552ab196e5d8 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,849 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e6577c5f177642aeabb0233d6b5e0f82 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,850 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/f693ef97d443479bbe8348015751e36e for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,850 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/1d460adc251c414d8f9e46ab583e599b for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,851 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/6e9a0b80488d4d17b6100d56d891937e for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,851 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 8ef507e9697a1fef706cfdf1625d1596 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 23:54:19,893 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:19,893 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:19,893 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:19,895 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,895 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:19,895 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:19,896 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:19,896 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:19,896 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:19,896 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:19,896 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:19,896 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:19,897 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:19,898 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-08 23:54:19,898 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:54:19,898 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:19,898 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:19,899 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:19,899 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:19,899 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:19,900 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:19,900 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:19,900 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:19,901 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:19,901 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:19,901 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:19,901 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:19,901 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:19,902 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:19,902 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,902 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:19,902 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:19,902 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:19,902 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,903 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,903 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,903 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:19,903 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:19,903 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 23:54:19,903 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:19,904 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:19,904 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900489904 2023-02-08 23:54:19,905 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:19,906 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:19,908 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:19,908 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:19,908 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:19,908 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900519908 2023-02-08 23:54:19,908 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900579908 2023-02-08 23:54:19,908 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:19,918 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=8ef507e9697a1fef706cfdf1625d1596, threads=14 2023-02-08 23:54:19,919 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/6e9a0b80488d4d17b6100d56d891937e for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,919 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/94dcf9c7e766468cacc6174fc3e6f340 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,919 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/a97350aa847344dfadd01fab3bff36a0 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,919 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e2c4cecb086e4e8eb2711f49b4425a4c for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,919 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e6577c5f177642aeabb0233d6b5e0f82 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,919 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/eccab892ccee400a9059552ab196e5d8 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,919 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/f693ef97d443479bbe8348015751e36e for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,920 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/bd061cdf305c4b5e852649a15c957abd for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,920 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/4f382ff1b13845d0b8a7073151178f1a for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,920 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/33258c4a961642df92d60ef8669f6c9a for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,920 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/11c46ab231134ad0bc579fbfe94b2f75 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,920 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/1d460adc251c414d8f9e46ab583e599b for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,921 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ee1bdb24106a42ccba67cd3dc436cf7c for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:19,921 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ecebd88d78864396909427b65832bf09 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,072 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/94dcf9c7e766468cacc6174fc3e6f340 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,083 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/6e9a0b80488d4d17b6100d56d891937e for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,083 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/33258c4a961642df92d60ef8669f6c9a for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,088 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e2c4cecb086e4e8eb2711f49b4425a4c for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,092 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ecebd88d78864396909427b65832bf09 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,106 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e6577c5f177642aeabb0233d6b5e0f82 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,116 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/11c46ab231134ad0bc579fbfe94b2f75 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,127 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/1d460adc251c414d8f9e46ab583e599b for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,136 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/4f382ff1b13845d0b8a7073151178f1a for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,136 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ee1bdb24106a42ccba67cd3dc436cf7c for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,141 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/f693ef97d443479bbe8348015751e36e for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,142 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/a97350aa847344dfadd01fab3bff36a0 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,143 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/eccab892ccee400a9059552ab196e5d8 for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,143 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/bd061cdf305c4b5e852649a15c957abd for region: 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:20,143 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 8ef507e9697a1fef706cfdf1625d1596 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 23:54:20,187 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:20,192 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/0a57f3ef37544e34b96c0446268af1c5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 23:54:20,194 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/1a75a4d55dc470081989e63e23308c9b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 23:54:20,194 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:20,194 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:20,195 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:20,301 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:54:20,409 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,409 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:20,409 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:20,409 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:20,410 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:20,410 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:20,410 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:20,410 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:20,410 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:20,411 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:20,412 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-08 23:54:20,412 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:54:20,412 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:20,412 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:20,412 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:20,413 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:20,413 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:20,413 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:20,413 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:20,414 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:20,414 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:20,414 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:20,415 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:20,415 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:20,415 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:20,415 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:20,416 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,416 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:20,416 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:20,419 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:20,419 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,420 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,420 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,420 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:20,420 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:20,420 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-08 23:54:20,421 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:20,421 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:20,422 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900490422 2023-02-08 23:54:20,422 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:20,423 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,426 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:20,426 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:20,426 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:20,426 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900520426 2023-02-08 23:54:20,426 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900580426 2023-02-08 23:54:20,426 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 23:54:20,429 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:20,433 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:20,433 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:20,433 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:20,677 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,677 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:20,677 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:20,677 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:20,678 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:20,678 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:20,678 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:20,678 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:20,678 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:20,679 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:20,680 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-08 23:54:20,680 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:54:20,680 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:20,680 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:20,680 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:20,681 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:20,681 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:20,681 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:20,681 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:20,681 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:20,682 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:20,682 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:20,682 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:20,682 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:20,682 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:20,683 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:20,683 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,683 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:20,683 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:20,683 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:20,684 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,684 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,684 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,684 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:20,684 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:20,684 DEBUG [Listener at localhost/37973] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,684 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 23:54:20,685 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:20,685 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:20,685 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900490685 2023-02-08 23:54:20,686 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:20,688 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,690 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:20,690 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:20,690 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:20,690 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900520690 2023-02-08 23:54:20,691 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900580691 2023-02-08 23:54:20,691 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 23:54:20,691 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:20,703 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:20,703 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 23:54:20,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:20,704 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:20,942 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,942 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:20,942 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:20,942 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:20,943 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:20,943 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:20,943 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:20,943 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:20,943 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:20,944 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:20,945 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-08 23:54:20,946 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:54:20,946 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:20,946 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:20,946 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:20,947 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:20,947 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:20,947 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:20,947 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:20,948 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:20,948 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:20,948 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:20,948 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:20,949 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:20,949 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:20,949 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:20,950 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,950 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:20,950 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:20,950 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:20,950 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,950 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,950 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,950 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:20,950 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:20,950 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 23:54:20,951 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:20,952 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:20,952 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900490952 2023-02-08 23:54:20,955 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:20,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:20,958 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:20,958 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:20,958 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:20,958 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900520958 2023-02-08 23:54:20,958 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900580958 2023-02-08 23:54:20,958 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:20,958 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:20,961 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:20,961 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 23:54:20,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:20,961 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:21,209 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,210 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:21,210 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:21,210 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:21,210 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:21,210 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:21,210 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:21,210 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:21,210 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:21,211 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:21,212 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-08 23:54:21,212 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:54:21,212 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:21,212 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:21,213 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:21,213 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:21,214 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:21,214 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:21,214 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:21,214 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:21,214 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:21,215 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:21,215 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:21,215 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:21,215 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:21,216 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:21,216 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,216 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:21,217 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:21,217 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:21,217 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,217 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,217 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,217 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:21,217 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:21,217 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 23:54:21,218 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:21,218 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:21,218 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900491218 2023-02-08 23:54:21,219 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:21,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,223 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:21,223 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:21,223 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:21,223 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900521223 2023-02-08 23:54:21,223 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900581223 2023-02-08 23:54:21,223 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:21,223 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:21,225 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:21,225 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 23:54:21,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:21,225 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:21,473 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,473 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:21,474 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:21,474 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:21,474 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:21,474 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:21,474 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:21,474 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:21,474 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:21,475 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:21,476 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-08 23:54:21,476 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:54:21,476 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:21,476 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:21,476 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:21,477 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:21,477 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:21,477 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:21,477 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:21,477 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:21,477 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:21,478 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:21,478 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:21,478 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:21,478 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:21,478 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:21,479 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,479 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:21,479 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:21,479 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:21,479 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,479 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,479 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,479 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:21,480 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:21,480 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 23:54:21,480 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:21,481 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:21,481 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900491481 2023-02-08 23:54:21,483 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:21,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,485 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:21,485 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:21,485 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:21,485 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900521485 2023-02-08 23:54:21,486 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900581486 2023-02-08 23:54:21,486 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 23:54:21,486 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:21,489 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN] 2023-02-08 23:54:21,489 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:21,489 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 23:54:21,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:21,489 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:21,736 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,736 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:21,736 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:21,736 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:21,737 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:21,737 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:21,737 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:21,737 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:21,737 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:21,738 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:21,739 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-08 23:54:21,739 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:54:21,739 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:54:21,739 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 23:54:21,739 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:21,740 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:21,740 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:21,740 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:21,741 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:21,741 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:21,741 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:21,741 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:21,742 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:21,742 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:21,742 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:21,742 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:21,743 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:21,744 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,744 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:21,744 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:21,744 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:21,744 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:21,744 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,744 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,745 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,745 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:21,745 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:21,745 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:21,745 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 23:54:21,746 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:21,746 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:21,746 INFO [Listener at localhost/37973] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596 to restore RIT 2023-02-08 23:54:21,747 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900491747 2023-02-08 23:54:21,748 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:21,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:21,749 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:21,752 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:21,752 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:21,752 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:21,752 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900521752 2023-02-08 23:54:21,752 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900581752 2023-02-08 23:54:21,752 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-08 23:54:21,752 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:21,757 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:21,757 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 23:54:21,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:21,757 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:22,002 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,002 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:22,003 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:22,003 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:22,003 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:22,003 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:22,003 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:22,003 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:22,003 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:22,004 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:22,005 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 23:54:22,005 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 23:54:22,005 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:22,005 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:22,005 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:22,005 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:22,006 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:22,006 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:22,006 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:22,006 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:22,006 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:22,007 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:22,007 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:22,007 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:22,007 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:22,008 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,008 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:22,008 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:22,008 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,008 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,008 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,008 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:22,008 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 23:54:22,009 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:22,009 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:22,009 INFO [Listener at localhost/37973] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596 to restore RIT 2023-02-08 23:54:22,010 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900492010 2023-02-08 23:54:22,011 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:22,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,013 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:22,014 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:22,015 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:22,015 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:22,015 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900522015 2023-02-08 23:54:22,015 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900582015 2023-02-08 23:54:22,015 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:54:22,015 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:22,017 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:22,017 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 23:54:22,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:22,017 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:22,265 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,265 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:22,265 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:22,265 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:22,266 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:22,266 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:22,266 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:22,266 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:22,266 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:22,267 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:22,267 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-08 23:54:22,267 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 23:54:22,268 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:22,268 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:22,268 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:22,268 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:22,269 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:22,269 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:22,269 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:22,269 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:22,269 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:22,269 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:22,270 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:22,270 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:22,270 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:22,270 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,270 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:22,270 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:22,270 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,271 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,271 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,271 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:22,271 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 23:54:22,272 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:22,272 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:22,272 INFO [Listener at localhost/37973] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596 to restore RIT 2023-02-08 23:54:22,272 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900492272 2023-02-08 23:54:22,273 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:22,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,275 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:22,277 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:22,277 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:22,278 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:22,278 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900522278 2023-02-08 23:54:22,278 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900582278 2023-02-08 23:54:22,278 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 23:54:22,278 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:22,280 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:22,280 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 23:54:22,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:22,280 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:22,528 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,528 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:22,528 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:22,528 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:22,529 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:22,529 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:22,529 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:22,529 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:22,529 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:22,530 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:22,531 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-08 23:54:22,531 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 23:54:22,531 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:22,531 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:22,531 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:22,532 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:22,532 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:22,532 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:22,532 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:22,532 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:22,533 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:22,533 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:22,533 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:22,533 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:22,533 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:22,534 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,534 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:22,534 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:22,534 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,534 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,534 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,534 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:54:22,534 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 23:54:22,535 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:22,535 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:22,535 INFO [Listener at localhost/37973] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596 to restore RIT 2023-02-08 23:54:22,536 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900492536 2023-02-08 23:54:22,536 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:22,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,540 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:22,542 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:22,542 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:22,542 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:22,542 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900522542 2023-02-08 23:54:22,542 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900582542 2023-02-08 23:54:22,543 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-08 23:54:22,543 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:54:22,545 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b exec-time=4.7570 sec 2023-02-08 23:54:22,545 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:22,546 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=true 2023-02-08 23:54:22,696 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:54:22,698 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=8ef507e9697a1fef706cfdf1625d1596, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:22,698 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900462697"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900462697"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900462697"}]},"ts":"1675900462697"} 2023-02-08 23:54:22,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:22,700 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:54:22,700 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:22,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:54:22,700 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:54:22,793 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:22,793 WARN [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-08 23:54:22,793 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:54:22,793 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:54:22,793 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:54:22,795 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:54:22,796 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:54:22,796 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:54:22,796 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:54:22,796 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:54:22,797 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:54:22,797 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-08 23:54:22,798 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:54:22,798 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:54:22,798 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:54:22,798 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:54:22,798 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:22,799 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:54:22,799 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:22,799 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:54:22,799 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:22,799 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:54:22,799 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:22,800 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:54:22,800 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:22,800 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:54:22,800 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:22,800 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:54:22,800 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:22,801 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-08 23:54:22,801 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:54:22,801 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:54:22,802 INFO [Listener at localhost/37973] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596 to restore RIT 2023-02-08 23:54:22,802 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900492802 2023-02-08 23:54:22,803 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:54:22,803 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:54:22,805 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:22,805 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8ef507e9697a1fef706cfdf1625d1596, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:54:22,805 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:54:22,805 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900522805 2023-02-08 23:54:22,805 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900582805 2023-02-08 23:54:22,805 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 23:54:22,805 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=8ef507e9697a1fef706cfdf1625d1596, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:22,806 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900462805"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900462805"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900462805"}]},"ts":"1675900462805"} 2023-02-08 23:54:22,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:22,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:22,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8ef507e9697a1fef706cfdf1625d1596, NAME => 'testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:22,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:22,963 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:22,963 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:22,963 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:22,963 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:22,964 INFO [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:22,965 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1 2023-02-08 23:54:22,965 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1 2023-02-08 23:54:22,966 INFO [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8ef507e9697a1fef706cfdf1625d1596 columnFamilyName cf1 2023-02-08 23:54:22,972 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/11c46ab231134ad0bc579fbfe94b2f75 2023-02-08 23:54:22,976 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/1d460adc251c414d8f9e46ab583e599b 2023-02-08 23:54:22,979 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/33258c4a961642df92d60ef8669f6c9a 2023-02-08 23:54:22,982 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/4f382ff1b13845d0b8a7073151178f1a 2023-02-08 23:54:22,985 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/bd061cdf305c4b5e852649a15c957abd 2023-02-08 23:54:22,988 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ecebd88d78864396909427b65832bf09 2023-02-08 23:54:22,992 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ee1bdb24106a42ccba67cd3dc436cf7c 2023-02-08 23:54:22,992 INFO [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(310): Store=8ef507e9697a1fef706cfdf1625d1596/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:22,992 INFO [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:22,993 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2 2023-02-08 23:54:22,993 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2 2023-02-08 23:54:22,994 INFO [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8ef507e9697a1fef706cfdf1625d1596 columnFamilyName cf2 2023-02-08 23:54:23,001 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/6e9a0b80488d4d17b6100d56d891937e 2023-02-08 23:54:23,005 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/94dcf9c7e766468cacc6174fc3e6f340 2023-02-08 23:54:23,009 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/a97350aa847344dfadd01fab3bff36a0 2023-02-08 23:54:23,013 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e2c4cecb086e4e8eb2711f49b4425a4c 2023-02-08 23:54:23,017 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e6577c5f177642aeabb0233d6b5e0f82 2023-02-08 23:54:23,021 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/eccab892ccee400a9059552ab196e5d8 2023-02-08 23:54:23,025 DEBUG [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/f693ef97d443479bbe8348015751e36e 2023-02-08 23:54:23,025 INFO [StoreOpener-8ef507e9697a1fef706cfdf1625d1596-1] regionserver.HStore(310): Store=8ef507e9697a1fef706cfdf1625d1596/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:23,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:23,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:23,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.splits 2023-02-08 23:54:23,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:23,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:23,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8ef507e9697a1fef706cfdf1625d1596; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64699622, jitterRate=-0.035900503396987915}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:23,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8ef507e9697a1fef706cfdf1625d1596: Running coprocessor pre-open hook at 1675900462963Writing region info on filesystem at 1675900462963Initializing all the Stores at 1675900462964 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900462964Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900462964Cleaning up temporary data from old regions at 1675900463027 (+63 ms)Cleaning up detritus from prior splits at 1675900463028 (+1 ms)Running coprocessor post-open hooks at 1675900463031 (+3 ms)Region opened successfully at 1675900463031 2023-02-08 23:54:23,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596., pid=48, masterSystemTime=1675900462959 2023-02-08 23:54:23,032 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:23,033 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:54:23,034 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-08 23:54:23,034 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1912): 8ef507e9697a1fef706cfdf1625d1596/cf1 is initiating minor compaction (all files) 2023-02-08 23:54:23,034 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8ef507e9697a1fef706cfdf1625d1596/cf1 in testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:23,034 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ee1bdb24106a42ccba67cd3dc436cf7c, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/bd061cdf305c4b5e852649a15c957abd, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ecebd88d78864396909427b65832bf09, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/11c46ab231134ad0bc579fbfe94b2f75, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/33258c4a961642df92d60ef8669f6c9a, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/1d460adc251c414d8f9e46ab583e599b, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/4f382ff1b13845d0b8a7073151178f1a] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp, totalSize=34.9 K 2023-02-08 23:54:23,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:54:23,034 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:54:23,034 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-08 23:54:23,035 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1912): 8ef507e9697a1fef706cfdf1625d1596/cf2 is initiating minor compaction (all files) 2023-02-08 23:54:23,035 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting ee1bdb24106a42ccba67cd3dc436cf7c, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900456326 2023-02-08 23:54:23,035 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8ef507e9697a1fef706cfdf1625d1596/cf2 in testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:23,035 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/a97350aa847344dfadd01fab3bff36a0, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/6e9a0b80488d4d17b6100d56d891937e, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e6577c5f177642aeabb0233d6b5e0f82, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/f693ef97d443479bbe8348015751e36e, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/94dcf9c7e766468cacc6174fc3e6f340, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/eccab892ccee400a9059552ab196e5d8, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e2c4cecb086e4e8eb2711f49b4425a4c] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp, totalSize=34.9 K 2023-02-08 23:54:23,035 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting bd061cdf305c4b5e852649a15c957abd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900456524 2023-02-08 23:54:23,035 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting a97350aa847344dfadd01fab3bff36a0, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900456326 2023-02-08 23:54:23,035 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting ecebd88d78864396909427b65832bf09, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900456726 2023-02-08 23:54:23,037 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting 6e9a0b80488d4d17b6100d56d891937e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900456524 2023-02-08 23:54:23,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:23,037 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 11c46ab231134ad0bc579fbfe94b2f75, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900456938 2023-02-08 23:54:23,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:23,037 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting e6577c5f177642aeabb0233d6b5e0f82, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900456726 2023-02-08 23:54:23,037 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 33258c4a961642df92d60ef8669f6c9a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900457129 2023-02-08 23:54:23,037 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=8ef507e9697a1fef706cfdf1625d1596, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:23,037 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting f693ef97d443479bbe8348015751e36e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900456938 2023-02-08 23:54:23,038 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900463037"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900463037"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900463037"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900463037"}]},"ts":"1675900463037"} 2023-02-08 23:54:23,038 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 1d460adc251c414d8f9e46ab583e599b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900457320 2023-02-08 23:54:23,038 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting 94dcf9c7e766468cacc6174fc3e6f340, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900457129 2023-02-08 23:54:23,038 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 4f382ff1b13845d0b8a7073151178f1a, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675900457526 2023-02-08 23:54:23,038 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting eccab892ccee400a9059552ab196e5d8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900457320 2023-02-08 23:54:23,039 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting e2c4cecb086e4e8eb2711f49b4425a4c, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675900457526 2023-02-08 23:54:23,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-08 23:54:23,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 in 233 msec 2023-02-08 23:54:23,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN in 1.5550 sec 2023-02-08 23:54:23,056 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-08 23:54:23,059 DEBUG [Listener at localhost/37973] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-08 23:54:23,060 INFO [Listener at localhost/37973] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-08 23:54:23,062 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8ef507e9697a1fef706cfdf1625d1596#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:23,062 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] throttle.PressureAwareThroughputController(145): 8ef507e9697a1fef706cfdf1625d1596#cf2#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:23,064 INFO [Listener at localhost/37973] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-08 23:54:23,065 INFO [Listener at localhost/37973] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-08 23:54:23,065 INFO [Listener at localhost/37973] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-08 23:54:23,081 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf2/12901996030c4b8dacdc9196f2d2811c as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/12901996030c4b8dacdc9196f2d2811c 2023-02-08 23:54:23,081 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/.tmp/cf1/d261b8c8243f42efb706818751645b1f as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/d261b8c8243f42efb706818751645b1f 2023-02-08 23:54:23,091 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8ef507e9697a1fef706cfdf1625d1596/cf1 of 8ef507e9697a1fef706cfdf1625d1596 into d261b8c8243f42efb706818751645b1f(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:23,091 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8ef507e9697a1fef706cfdf1625d1596: 2023-02-08 23:54:23,091 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596., storeName=8ef507e9697a1fef706cfdf1625d1596/cf1, priority=9, startTime=1675900463032; duration=0sec 2023-02-08 23:54:23,093 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8ef507e9697a1fef706cfdf1625d1596/cf2 of 8ef507e9697a1fef706cfdf1625d1596 into 12901996030c4b8dacdc9196f2d2811c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:23,093 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:23,093 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8ef507e9697a1fef706cfdf1625d1596: 2023-02-08 23:54:23,093 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596., storeName=8ef507e9697a1fef706cfdf1625d1596/cf2, priority=9, startTime=1675900463034; duration=0sec 2023-02-08 23:54:23,093 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:23,105 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:23,105 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:23,105 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:23,107 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:23,107 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:23,107 INFO [Listener at localhost/37973] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-08 23:54:23,107 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-08 23:54:23,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:23,111 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:23,111 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:23,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-08 23:54:23,112 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900463112"}]},"ts":"1675900463112"} 2023-02-08 23:54:23,113 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-08 23:54:23,115 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-08 23:54:23,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN}] 2023-02-08 23:54:23,116 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN 2023-02-08 23:54:23,117 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:23,117 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900463117"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900463117"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900463117"}]},"ts":"1675900463117"} 2023-02-08 23:54:23,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:23,212 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:23,212 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:23,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-08 23:54:23,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:23,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8ef507e9697a1fef706cfdf1625d1596, disabling compactions & flushes 2023-02-08 23:54:23,271 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:23,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:23,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. after waiting 0 ms 2023-02-08 23:54:23,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:23,271 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ee1bdb24106a42ccba67cd3dc436cf7c, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/bd061cdf305c4b5e852649a15c957abd, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ecebd88d78864396909427b65832bf09, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/11c46ab231134ad0bc579fbfe94b2f75, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/33258c4a961642df92d60ef8669f6c9a, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/1d460adc251c414d8f9e46ab583e599b, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/4f382ff1b13845d0b8a7073151178f1a] to archive 2023-02-08 23:54:23,272 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:23,275 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ee1bdb24106a42ccba67cd3dc436cf7c to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ee1bdb24106a42ccba67cd3dc436cf7c 2023-02-08 23:54:23,276 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/bd061cdf305c4b5e852649a15c957abd to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/bd061cdf305c4b5e852649a15c957abd 2023-02-08 23:54:23,278 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ecebd88d78864396909427b65832bf09 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/ecebd88d78864396909427b65832bf09 2023-02-08 23:54:23,279 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/11c46ab231134ad0bc579fbfe94b2f75 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/11c46ab231134ad0bc579fbfe94b2f75 2023-02-08 23:54:23,281 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/33258c4a961642df92d60ef8669f6c9a to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/33258c4a961642df92d60ef8669f6c9a 2023-02-08 23:54:23,282 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/1d460adc251c414d8f9e46ab583e599b to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/1d460adc251c414d8f9e46ab583e599b 2023-02-08 23:54:23,283 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/4f382ff1b13845d0b8a7073151178f1a to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/4f382ff1b13845d0b8a7073151178f1a 2023-02-08 23:54:23,288 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/a97350aa847344dfadd01fab3bff36a0, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/6e9a0b80488d4d17b6100d56d891937e, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e6577c5f177642aeabb0233d6b5e0f82, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/f693ef97d443479bbe8348015751e36e, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/94dcf9c7e766468cacc6174fc3e6f340, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/eccab892ccee400a9059552ab196e5d8, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e2c4cecb086e4e8eb2711f49b4425a4c] to archive 2023-02-08 23:54:23,289 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:23,291 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/a97350aa847344dfadd01fab3bff36a0 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/a97350aa847344dfadd01fab3bff36a0 2023-02-08 23:54:23,292 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/6e9a0b80488d4d17b6100d56d891937e to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/6e9a0b80488d4d17b6100d56d891937e 2023-02-08 23:54:23,293 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e6577c5f177642aeabb0233d6b5e0f82 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e6577c5f177642aeabb0233d6b5e0f82 2023-02-08 23:54:23,295 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/f693ef97d443479bbe8348015751e36e to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/f693ef97d443479bbe8348015751e36e 2023-02-08 23:54:23,296 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/94dcf9c7e766468cacc6174fc3e6f340 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/94dcf9c7e766468cacc6174fc3e6f340 2023-02-08 23:54:23,297 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/eccab892ccee400a9059552ab196e5d8 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/eccab892ccee400a9059552ab196e5d8 2023-02-08 23:54:23,299 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e2c4cecb086e4e8eb2711f49b4425a4c to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/e2c4cecb086e4e8eb2711f49b4425a4c 2023-02-08 23:54:23,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 23:54:23,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:23,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596. 2023-02-08 23:54:23,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8ef507e9697a1fef706cfdf1625d1596: Waiting for close lock at 1675900463271Running coprocessor pre-close hooks at 1675900463271Disabling compacts and flushes for region at 1675900463271Disabling writes for close at 1675900463271Writing region close event to WAL at 1675900463300 (+29 ms)Running coprocessor post-close hooks at 1675900463304 (+4 ms)Closed at 1675900463304 2023-02-08 23:54:23,306 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:23,306 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=8ef507e9697a1fef706cfdf1625d1596, regionState=CLOSED 2023-02-08 23:54:23,307 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900463306"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900463306"}]},"ts":"1675900463306"} 2023-02-08 23:54:23,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-08 23:54:23,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 8ef507e9697a1fef706cfdf1625d1596, server=jenkins-hbase4.apache.org,36199,1675900441120 in 190 msec 2023-02-08 23:54:23,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-08 23:54:23,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, UNASSIGN in 194 msec 2023-02-08 23:54:23,312 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900463312"}]},"ts":"1675900463312"} 2023-02-08 23:54:23,313 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-08 23:54:23,315 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-08 23:54:23,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 207 msec 2023-02-08 23:54:23,413 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:23,413 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:23,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-08 23:54:23,415 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-08 23:54:23,415 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:23,415 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:23,416 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-08 23:54:23,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:23,419 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:23,419 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:23,419 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:23,419 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:23,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-08 23:54:23,423 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:23,425 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/recovered.edits] 2023-02-08 23:54:23,428 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/d261b8c8243f42efb706818751645b1f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf1/d261b8c8243f42efb706818751645b1f 2023-02-08 23:54:23,430 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/12901996030c4b8dacdc9196f2d2811c to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/cf2/12901996030c4b8dacdc9196f2d2811c 2023-02-08 23:54:23,433 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/recovered.edits/90.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596/recovered.edits/90.seqid 2023-02-08 23:54:23,433 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackAndDoubleExecution/8ef507e9697a1fef706cfdf1625d1596 2023-02-08 23:54:23,433 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-08 23:54:23,437 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:23,438 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-08 23:54:23,440 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-08 23:54:23,441 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:23,441 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-08 23:54:23,441 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900463441"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:23,443 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-08 23:54:23,443 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 8ef507e9697a1fef706cfdf1625d1596, NAME => 'testRollbackAndDoubleExecution,,1675900455821.8ef507e9697a1fef706cfdf1625d1596.', STARTKEY => '', ENDKEY => ''}] 2023-02-08 23:54:23,443 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-08 23:54:23,443 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675900463443"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:23,445 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-08 23:54:23,447 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:54:23,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 31 msec 2023-02-08 23:54:23,520 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:23,520 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:23,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-08 23:54:23,522 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-08 23:54:23,536 INFO [Listener at localhost/37973] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=397 (was 400), OpenFileDescriptor=849 (was 855), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=268 (was 264) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3362 (was 3383) 2023-02-08 23:54:23,550 INFO [Listener at localhost/37973] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=397, OpenFileDescriptor=849, MaxFileDescriptor=60000, SystemLoadAverage=268, ProcessCount=168, AvailableMemoryMB=3362 2023-02-08 23:54:23,550 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:23,550 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:23,550 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:23,554 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 23:54:23,554 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 23:54:23,556 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:54:23,558 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:54:23,558 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:54:23,560 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:23,561 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd empty. 2023-02-08 23:54:23,561 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:23,561 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-08 23:54:23,574 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:23,575 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => a2b657ee6899d54a60f2b2b3f0c2f7fd, NAME => 'testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp 2023-02-08 23:54:23,585 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:23,585 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing a2b657ee6899d54a60f2b2b3f0c2f7fd, disabling compactions & flushes 2023-02-08 23:54:23,585 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. 2023-02-08 23:54:23,585 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. 2023-02-08 23:54:23,585 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. after waiting 0 ms 2023-02-08 23:54:23,585 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. 2023-02-08 23:54:23,585 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. 2023-02-08 23:54:23,585 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for a2b657ee6899d54a60f2b2b3f0c2f7fd: Waiting for close lock at 1675900463585Disabling compacts and flushes for region at 1675900463585Disabling writes for close at 1675900463585Writing region close event to WAL at 1675900463585Closed at 1675900463585 2023-02-08 23:54:23,588 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:54:23,588 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675900463588"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900463588"}]},"ts":"1675900463588"} 2023-02-08 23:54:23,590 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:54:23,591 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:54:23,591 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900463591"}]},"ts":"1675900463591"} 2023-02-08 23:54:23,592 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-08 23:54:23,595 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:54:23,596 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:54:23,596 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:54:23,596 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:54:23,596 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 23:54:23,597 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a2b657ee6899d54a60f2b2b3f0c2f7fd, ASSIGN}] 2023-02-08 23:54:23,598 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a2b657ee6899d54a60f2b2b3f0c2f7fd, ASSIGN 2023-02-08 23:54:23,599 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a2b657ee6899d54a60f2b2b3f0c2f7fd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=false 2023-02-08 23:54:23,749 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:54:23,750 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=a2b657ee6899d54a60f2b2b3f0c2f7fd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:23,750 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675900463750"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900463750"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900463750"}]},"ts":"1675900463750"} 2023-02-08 23:54:23,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure a2b657ee6899d54a60f2b2b3f0c2f7fd, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:23,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. 2023-02-08 23:54:23,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a2b657ee6899d54a60f2b2b3f0c2f7fd, NAME => 'testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:23,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:23,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:23,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:23,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:23,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:23,909 INFO [StoreOpener-a2b657ee6899d54a60f2b2b3f0c2f7fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:23,910 DEBUG [StoreOpener-a2b657ee6899d54a60f2b2b3f0c2f7fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd/cf1 2023-02-08 23:54:23,910 DEBUG [StoreOpener-a2b657ee6899d54a60f2b2b3f0c2f7fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd/cf1 2023-02-08 23:54:23,911 INFO [StoreOpener-a2b657ee6899d54a60f2b2b3f0c2f7fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2b657ee6899d54a60f2b2b3f0c2f7fd columnFamilyName cf1 2023-02-08 23:54:23,911 INFO [StoreOpener-a2b657ee6899d54a60f2b2b3f0c2f7fd-1] regionserver.HStore(310): Store=a2b657ee6899d54a60f2b2b3f0c2f7fd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:23,911 INFO [StoreOpener-a2b657ee6899d54a60f2b2b3f0c2f7fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:23,913 DEBUG [StoreOpener-a2b657ee6899d54a60f2b2b3f0c2f7fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd/cf2 2023-02-08 23:54:23,913 DEBUG [StoreOpener-a2b657ee6899d54a60f2b2b3f0c2f7fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd/cf2 2023-02-08 23:54:23,913 INFO [StoreOpener-a2b657ee6899d54a60f2b2b3f0c2f7fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2b657ee6899d54a60f2b2b3f0c2f7fd columnFamilyName cf2 2023-02-08 23:54:23,913 INFO [StoreOpener-a2b657ee6899d54a60f2b2b3f0c2f7fd-1] regionserver.HStore(310): Store=a2b657ee6899d54a60f2b2b3f0c2f7fd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:23,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:23,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:23,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:23,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:23,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:23,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a2b657ee6899d54a60f2b2b3f0c2f7fd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63104594, jitterRate=-0.059668272733688354}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:23,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a2b657ee6899d54a60f2b2b3f0c2f7fd: Running coprocessor pre-open hook at 1675900463908Writing region info on filesystem at 1675900463908Initializing all the Stores at 1675900463909 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900463909Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900463909Cleaning up temporary data from old regions at 1675900463915 (+6 ms)Cleaning up detritus from prior splits at 1675900463916 (+1 ms)Running coprocessor post-open hooks at 1675900463920 (+4 ms)Region opened successfully at 1675900463920 2023-02-08 23:54:23,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd., pid=55, masterSystemTime=1675900463904 2023-02-08 23:54:23,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. 2023-02-08 23:54:23,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. 2023-02-08 23:54:23,923 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=a2b657ee6899d54a60f2b2b3f0c2f7fd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:23,923 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675900463923"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900463923"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900463923"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900463923"}]},"ts":"1675900463923"} 2023-02-08 23:54:23,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-08 23:54:23,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure a2b657ee6899d54a60f2b2b3f0c2f7fd, server=jenkins-hbase4.apache.org,36199,1675900441120 in 172 msec 2023-02-08 23:54:23,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-08 23:54:23,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a2b657ee6899d54a60f2b2b3f0c2f7fd, ASSIGN in 329 msec 2023-02-08 23:54:23,928 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:54:23,928 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900463928"}]},"ts":"1675900463928"} 2023-02-08 23:54:23,930 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-08 23:54:23,933 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:54:23,935 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 378 msec 2023-02-08 23:54:24,057 DEBUG [Listener at localhost/37973] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:24,058 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,059 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,059 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,059 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a2b657ee6899d54a60f2b2b3f0c2f7fd, UNASSIGN}] 2023-02-08 23:54:24,062 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a2b657ee6899d54a60f2b2b3f0c2f7fd, UNASSIGN 2023-02-08 23:54:24,063 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=a2b657ee6899d54a60f2b2b3f0c2f7fd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:24,063 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675900464063"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900464063"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900464063"}]},"ts":"1675900464063"} 2023-02-08 23:54:24,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure a2b657ee6899d54a60f2b2b3f0c2f7fd, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:24,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:24,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a2b657ee6899d54a60f2b2b3f0c2f7fd, disabling compactions & flushes 2023-02-08 23:54:24,217 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. 2023-02-08 23:54:24,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. 2023-02-08 23:54:24,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. after waiting 0 ms 2023-02-08 23:54:24,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. 2023-02-08 23:54:24,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-08 23:54:24,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:24,221 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd. 2023-02-08 23:54:24,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a2b657ee6899d54a60f2b2b3f0c2f7fd: Waiting for close lock at 1675900464216Running coprocessor pre-close hooks at 1675900464216Disabling compacts and flushes for region at 1675900464216Disabling writes for close at 1675900464217 (+1 ms)Writing region close event to WAL at 1675900464218 (+1 ms)Running coprocessor post-close hooks at 1675900464221 (+3 ms)Closed at 1675900464221 2023-02-08 23:54:24,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:24,223 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=a2b657ee6899d54a60f2b2b3f0c2f7fd, regionState=CLOSED 2023-02-08 23:54:24,223 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675900464223"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900464223"}]},"ts":"1675900464223"} 2023-02-08 23:54:24,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-08 23:54:24,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure a2b657ee6899d54a60f2b2b3f0c2f7fd, server=jenkins-hbase4.apache.org,36199,1675900441120 in 161 msec 2023-02-08 23:54:24,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-08 23:54:24,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a2b657ee6899d54a60f2b2b3f0c2f7fd, UNASSIGN in 165 msec 2023-02-08 23:54:24,258 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-08 23:54:24,260 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-08 23:54:24,261 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675900464261"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1675900464261"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1675900464261"}]},"ts":"1675900464261"} 2023-02-08 23:54:24,261 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900464261"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900464261"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900464261"}]},"ts":"1675900464261"} 2023-02-08 23:54:24,262 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900464261"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900464261"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900464261"}]},"ts":"1675900464261"} 2023-02-08 23:54:24,265 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2257771f8158addd99a8446b310dd3fe, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0a89e703978b35555b1c9da5e800b1a9, ASSIGN}] 2023-02-08 23:54:24,266 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0a89e703978b35555b1c9da5e800b1a9, ASSIGN 2023-02-08 23:54:24,266 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0a89e703978b35555b1c9da5e800b1a9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=false 2023-02-08 23:54:24,267 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2257771f8158addd99a8446b310dd3fe, ASSIGN 2023-02-08 23:54:24,268 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2257771f8158addd99a8446b310dd3fe, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=false 2023-02-08 23:54:24,417 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 23:54:24,418 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=2257771f8158addd99a8446b310dd3fe, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:24,419 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900464418"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900464418"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900464418"}]},"ts":"1675900464418"} 2023-02-08 23:54:24,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 2257771f8158addd99a8446b310dd3fe, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:24,423 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=0a89e703978b35555b1c9da5e800b1a9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:24,423 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900464423"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900464423"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900464423"}]},"ts":"1675900464423"} 2023-02-08 23:54:24,424 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 0a89e703978b35555b1c9da5e800b1a9, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:24,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9. 2023-02-08 23:54:24,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0a89e703978b35555b1c9da5e800b1a9, NAME => 'testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9.', STARTKEY => '41', ENDKEY => ''} 2023-02-08 23:54:24,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:24,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:24,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,577 INFO [StoreOpener-0a89e703978b35555b1c9da5e800b1a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,578 DEBUG [StoreOpener-0a89e703978b35555b1c9da5e800b1a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9/cf1 2023-02-08 23:54:24,578 DEBUG [StoreOpener-0a89e703978b35555b1c9da5e800b1a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9/cf1 2023-02-08 23:54:24,579 INFO [StoreOpener-0a89e703978b35555b1c9da5e800b1a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0a89e703978b35555b1c9da5e800b1a9 columnFamilyName cf1 2023-02-08 23:54:24,579 INFO [StoreOpener-0a89e703978b35555b1c9da5e800b1a9-1] regionserver.HStore(310): Store=0a89e703978b35555b1c9da5e800b1a9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:24,579 INFO [StoreOpener-0a89e703978b35555b1c9da5e800b1a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,580 DEBUG [StoreOpener-0a89e703978b35555b1c9da5e800b1a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9/cf2 2023-02-08 23:54:24,581 DEBUG [StoreOpener-0a89e703978b35555b1c9da5e800b1a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9/cf2 2023-02-08 23:54:24,581 INFO [StoreOpener-0a89e703978b35555b1c9da5e800b1a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0a89e703978b35555b1c9da5e800b1a9 columnFamilyName cf2 2023-02-08 23:54:24,581 INFO [StoreOpener-0a89e703978b35555b1c9da5e800b1a9-1] regionserver.HStore(310): Store=0a89e703978b35555b1c9da5e800b1a9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:24,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:24,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0a89e703978b35555b1c9da5e800b1a9; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67501680, jitterRate=0.005853414535522461}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:24,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0a89e703978b35555b1c9da5e800b1a9: Running coprocessor pre-open hook at 1675900464576Writing region info on filesystem at 1675900464576Initializing all the Stores at 1675900464577 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900464577Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900464577Cleaning up temporary data from old regions at 1675900464584 (+7 ms)Cleaning up detritus from prior splits at 1675900464584Running coprocessor post-open hooks at 1675900464587 (+3 ms)Region opened successfully at 1675900464587 2023-02-08 23:54:24,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9., pid=62, masterSystemTime=1675900464572 2023-02-08 23:54:24,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9. 2023-02-08 23:54:24,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9. 2023-02-08 23:54:24,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe. 2023-02-08 23:54:24,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2257771f8158addd99a8446b310dd3fe, NAME => 'testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe.', STARTKEY => '', ENDKEY => '41'} 2023-02-08 23:54:24,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:24,589 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=0a89e703978b35555b1c9da5e800b1a9, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:24,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 2257771f8158addd99a8446b310dd3fe 2023-02-08 23:54:24,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:24,590 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900464589"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900464589"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900464589"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900464589"}]},"ts":"1675900464589"} 2023-02-08 23:54:24,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2257771f8158addd99a8446b310dd3fe 2023-02-08 23:54:24,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2257771f8158addd99a8446b310dd3fe 2023-02-08 23:54:24,591 INFO [StoreOpener-2257771f8158addd99a8446b310dd3fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2257771f8158addd99a8446b310dd3fe 2023-02-08 23:54:24,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-08 23:54:24,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 0a89e703978b35555b1c9da5e800b1a9, server=jenkins-hbase4.apache.org,36199,1675900441120 in 167 msec 2023-02-08 23:54:24,593 DEBUG [StoreOpener-2257771f8158addd99a8446b310dd3fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe/cf1 2023-02-08 23:54:24,593 DEBUG [StoreOpener-2257771f8158addd99a8446b310dd3fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe/cf1 2023-02-08 23:54:24,593 INFO [StoreOpener-2257771f8158addd99a8446b310dd3fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2257771f8158addd99a8446b310dd3fe columnFamilyName cf1 2023-02-08 23:54:24,594 INFO [StoreOpener-2257771f8158addd99a8446b310dd3fe-1] regionserver.HStore(310): Store=2257771f8158addd99a8446b310dd3fe/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:24,594 INFO [StoreOpener-2257771f8158addd99a8446b310dd3fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2257771f8158addd99a8446b310dd3fe 2023-02-08 23:54:24,594 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0a89e703978b35555b1c9da5e800b1a9, ASSIGN in 328 msec 2023-02-08 23:54:24,595 DEBUG [StoreOpener-2257771f8158addd99a8446b310dd3fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe/cf2 2023-02-08 23:54:24,595 DEBUG [StoreOpener-2257771f8158addd99a8446b310dd3fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe/cf2 2023-02-08 23:54:24,595 INFO [StoreOpener-2257771f8158addd99a8446b310dd3fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2257771f8158addd99a8446b310dd3fe columnFamilyName cf2 2023-02-08 23:54:24,596 INFO [StoreOpener-2257771f8158addd99a8446b310dd3fe-1] regionserver.HStore(310): Store=2257771f8158addd99a8446b310dd3fe/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:24,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe 2023-02-08 23:54:24,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe 2023-02-08 23:54:24,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:24,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2257771f8158addd99a8446b310dd3fe 2023-02-08 23:54:24,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2257771f8158addd99a8446b310dd3fe; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71929814, jitterRate=0.07183775305747986}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:24,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2257771f8158addd99a8446b310dd3fe: Running coprocessor pre-open hook at 1675900464590Writing region info on filesystem at 1675900464590Initializing all the Stores at 1675900464591 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900464591Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900464591Cleaning up temporary data from old regions at 1675900464598 (+7 ms)Cleaning up detritus from prior splits at 1675900464599 (+1 ms)Running coprocessor post-open hooks at 1675900464602 (+3 ms)Region opened successfully at 1675900464602 2023-02-08 23:54:24,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe., pid=61, masterSystemTime=1675900464572 2023-02-08 23:54:24,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe. 2023-02-08 23:54:24,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe. 2023-02-08 23:54:24,604 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=2257771f8158addd99a8446b310dd3fe, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:24,604 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900464604"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900464604"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900464604"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900464604"}]},"ts":"1675900464604"} 2023-02-08 23:54:24,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-08 23:54:24,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 2257771f8158addd99a8446b310dd3fe, server=jenkins-hbase4.apache.org,36199,1675900441120 in 186 msec 2023-02-08 23:54:24,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-08 23:54:24,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2257771f8158addd99a8446b310dd3fe, ASSIGN in 342 msec 2023-02-08 23:54:24,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 in 552 msec 2023-02-08 23:54:24,812 DEBUG [Listener at localhost/37973] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 23:54:24,816 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:24,816 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:24,816 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:24,817 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:24,817 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:24,818 INFO [Listener at localhost/37973] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-08 23:54:24,818 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-08 23:54:24,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:54:24,820 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:24,821 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:24,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-08 23:54:24,821 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900464821"}]},"ts":"1675900464821"} 2023-02-08 23:54:24,822 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-08 23:54:24,824 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-08 23:54:24,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2257771f8158addd99a8446b310dd3fe, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0a89e703978b35555b1c9da5e800b1a9, UNASSIGN}] 2023-02-08 23:54:24,825 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0a89e703978b35555b1c9da5e800b1a9, UNASSIGN 2023-02-08 23:54:24,826 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=0a89e703978b35555b1c9da5e800b1a9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:24,826 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900464825"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900464825"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900464825"}]},"ts":"1675900464825"} 2023-02-08 23:54:24,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 0a89e703978b35555b1c9da5e800b1a9, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:24,829 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2257771f8158addd99a8446b310dd3fe, UNASSIGN 2023-02-08 23:54:24,830 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=2257771f8158addd99a8446b310dd3fe, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:24,830 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900464830"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900464830"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900464830"}]},"ts":"1675900464830"} 2023-02-08 23:54:24,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 2257771f8158addd99a8446b310dd3fe, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:24,921 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:24,922 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:24,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-08 23:54:24,979 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0a89e703978b35555b1c9da5e800b1a9, disabling compactions & flushes 2023-02-08 23:54:24,980 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9. 2023-02-08 23:54:24,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9. 2023-02-08 23:54:24,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9. after waiting 0 ms 2023-02-08 23:54:24,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9. 2023-02-08 23:54:24,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-08 23:54:24,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:24,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9. 2023-02-08 23:54:24,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0a89e703978b35555b1c9da5e800b1a9: Waiting for close lock at 1675900464979Running coprocessor pre-close hooks at 1675900464979Disabling compacts and flushes for region at 1675900464979Disabling writes for close at 1675900464980 (+1 ms)Writing region close event to WAL at 1675900464980Running coprocessor post-close hooks at 1675900464985 (+5 ms)Closed at 1675900464985 2023-02-08 23:54:24,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:24,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2257771f8158addd99a8446b310dd3fe 2023-02-08 23:54:24,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2257771f8158addd99a8446b310dd3fe, disabling compactions & flushes 2023-02-08 23:54:24,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe. 2023-02-08 23:54:24,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe. 2023-02-08 23:54:24,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe. after waiting 0 ms 2023-02-08 23:54:24,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe. 2023-02-08 23:54:24,988 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=0a89e703978b35555b1c9da5e800b1a9, regionState=CLOSED 2023-02-08 23:54:24,988 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900464988"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900464988"}]},"ts":"1675900464988"} 2023-02-08 23:54:24,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-08 23:54:24,992 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-08 23:54:24,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 0a89e703978b35555b1c9da5e800b1a9, server=jenkins-hbase4.apache.org,36199,1675900441120 in 163 msec 2023-02-08 23:54:24,992 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:24,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe. 2023-02-08 23:54:24,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2257771f8158addd99a8446b310dd3fe: Waiting for close lock at 1675900464987Running coprocessor pre-close hooks at 1675900464987Disabling compacts and flushes for region at 1675900464987Disabling writes for close at 1675900464988 (+1 ms)Writing region close event to WAL at 1675900464989 (+1 ms)Running coprocessor post-close hooks at 1675900464992 (+3 ms)Closed at 1675900464993 (+1 ms) 2023-02-08 23:54:24,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0a89e703978b35555b1c9da5e800b1a9, UNASSIGN in 168 msec 2023-02-08 23:54:24,994 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2257771f8158addd99a8446b310dd3fe 2023-02-08 23:54:24,994 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=2257771f8158addd99a8446b310dd3fe, regionState=CLOSED 2023-02-08 23:54:24,994 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900464994"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900464994"}]},"ts":"1675900464994"} 2023-02-08 23:54:24,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-08 23:54:24,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 2257771f8158addd99a8446b310dd3fe, server=jenkins-hbase4.apache.org,36199,1675900441120 in 164 msec 2023-02-08 23:54:24,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-02-08 23:54:24,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2257771f8158addd99a8446b310dd3fe, UNASSIGN in 173 msec 2023-02-08 23:54:24,999 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900464999"}]},"ts":"1675900464999"} 2023-02-08 23:54:25,000 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-08 23:54:25,002 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-08 23:54:25,003 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 184 msec 2023-02-08 23:54:25,123 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:25,123 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:25,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-08 23:54:25,124 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-08 23:54:25,124 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:25,124 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:25,125 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-08 23:54:25,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:54:25,128 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:54:25,128 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:25,128 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:25,128 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:54:25,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-08 23:54:25,134 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:25,134 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:25,134 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe 2023-02-08 23:54:25,136 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe/recovered.edits] 2023-02-08 23:54:25,136 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9/recovered.edits] 2023-02-08 23:54:25,136 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd/recovered.edits] 2023-02-08 23:54:25,142 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9/recovered.edits/7.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9/recovered.edits/7.seqid 2023-02-08 23:54:25,142 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd/recovered.edits/4.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd/recovered.edits/4.seqid 2023-02-08 23:54:25,143 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe/recovered.edits/7.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe/recovered.edits/7.seqid 2023-02-08 23:54:25,143 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:54:25,143 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/a2b657ee6899d54a60f2b2b3f0c2f7fd 2023-02-08 23:54:25,143 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionNoStoreFile/2257771f8158addd99a8446b310dd3fe 2023-02-08 23:54:25,143 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-08 23:54:25,145 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:54:25,147 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-08 23:54:25,149 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-08 23:54:25,150 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:54:25,150 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-08 23:54:25,150 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900465150"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:25,150 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900465150"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:25,150 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900465150"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:25,152 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 23:54:25,152 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a2b657ee6899d54a60f2b2b3f0c2f7fd, NAME => 'testSplitTableRegionNoStoreFile,,1675900463555.a2b657ee6899d54a60f2b2b3f0c2f7fd.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 2257771f8158addd99a8446b310dd3fe, NAME => 'testSplitTableRegionNoStoreFile,,1675900464057.2257771f8158addd99a8446b310dd3fe.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 0a89e703978b35555b1c9da5e800b1a9, NAME => 'testSplitTableRegionNoStoreFile,41,1675900464057.0a89e703978b35555b1c9da5e800b1a9.', STARTKEY => '41', ENDKEY => ''}] 2023-02-08 23:54:25,152 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-08 23:54:25,152 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675900465152"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:25,153 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-08 23:54:25,155 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:54:25,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 30 msec 2023-02-08 23:54:25,229 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:25,229 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:25,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-08 23:54:25,230 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-08 23:54:25,245 INFO [Listener at localhost/37973] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=401 (was 397) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1942556411_15 at /127.0.0.1:39338 [Waiting for operation #35] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1942556411_15 at /127.0.0.1:58436 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=849 (was 849), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=268 (was 268), ProcessCount=168 (was 168), AvailableMemoryMB=3355 (was 3362) 2023-02-08 23:54:25,258 INFO [Listener at localhost/37973] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=401, OpenFileDescriptor=849, MaxFileDescriptor=60000, SystemLoadAverage=268, ProcessCount=168, AvailableMemoryMB=3354 2023-02-08 23:54:25,259 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:25,259 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:25,259 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:25,262 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 23:54:25,262 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 23:54:25,263 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,265 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:54:25,266 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:54:25,267 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:25,268 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc empty. 2023-02-08 23:54:25,268 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:25,268 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-08 23:54:25,281 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:25,283 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0a59441954132eb44621ade02029d4dc, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp 2023-02-08 23:54:25,293 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:25,293 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 0a59441954132eb44621ade02029d4dc, disabling compactions & flushes 2023-02-08 23:54:25,293 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:25,293 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:25,293 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. after waiting 0 ms 2023-02-08 23:54:25,293 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:25,293 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:25,293 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 0a59441954132eb44621ade02029d4dc: Waiting for close lock at 1675900465293Disabling compacts and flushes for region at 1675900465293Disabling writes for close at 1675900465293Writing region close event to WAL at 1675900465293Closed at 1675900465293 2023-02-08 23:54:25,295 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:54:25,296 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675900465296"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900465296"}]},"ts":"1675900465296"} 2023-02-08 23:54:25,297 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:54:25,298 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:54:25,298 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900465298"}]},"ts":"1675900465298"} 2023-02-08 23:54:25,299 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-08 23:54:25,302 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:54:25,303 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:54:25,303 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:54:25,303 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:54:25,303 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 23:54:25,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0a59441954132eb44621ade02029d4dc, ASSIGN}] 2023-02-08 23:54:25,305 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0a59441954132eb44621ade02029d4dc, ASSIGN 2023-02-08 23:54:25,307 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0a59441954132eb44621ade02029d4dc, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=false 2023-02-08 23:54:25,457 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:54:25,458 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=0a59441954132eb44621ade02029d4dc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,458 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675900465458"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900465458"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900465458"}]},"ts":"1675900465458"} 2023-02-08 23:54:25,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 0a59441954132eb44621ade02029d4dc, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:25,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:25,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0a59441954132eb44621ade02029d4dc, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:25,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:25,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:25,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:25,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:25,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:25,617 INFO [StoreOpener-0a59441954132eb44621ade02029d4dc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:25,618 DEBUG [StoreOpener-0a59441954132eb44621ade02029d4dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1 2023-02-08 23:54:25,618 DEBUG [StoreOpener-0a59441954132eb44621ade02029d4dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1 2023-02-08 23:54:25,618 INFO [StoreOpener-0a59441954132eb44621ade02029d4dc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0a59441954132eb44621ade02029d4dc columnFamilyName cf1 2023-02-08 23:54:25,619 INFO [StoreOpener-0a59441954132eb44621ade02029d4dc-1] regionserver.HStore(310): Store=0a59441954132eb44621ade02029d4dc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:25,619 INFO [StoreOpener-0a59441954132eb44621ade02029d4dc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:25,620 DEBUG [StoreOpener-0a59441954132eb44621ade02029d4dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2 2023-02-08 23:54:25,620 DEBUG [StoreOpener-0a59441954132eb44621ade02029d4dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2 2023-02-08 23:54:25,621 INFO [StoreOpener-0a59441954132eb44621ade02029d4dc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0a59441954132eb44621ade02029d4dc columnFamilyName cf2 2023-02-08 23:54:25,621 INFO [StoreOpener-0a59441954132eb44621ade02029d4dc-1] regionserver.HStore(310): Store=0a59441954132eb44621ade02029d4dc/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:25,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:25,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:25,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:25,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:25,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:25,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0a59441954132eb44621ade02029d4dc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60499519, jitterRate=-0.09848691523075104}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:25,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0a59441954132eb44621ade02029d4dc: Running coprocessor pre-open hook at 1675900465616Writing region info on filesystem at 1675900465616Initializing all the Stores at 1675900465616Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900465616Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900465617 (+1 ms)Cleaning up temporary data from old regions at 1675900465623 (+6 ms)Cleaning up detritus from prior splits at 1675900465623Running coprocessor post-open hooks at 1675900465628 (+5 ms)Region opened successfully at 1675900465628 2023-02-08 23:54:25,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc., pid=71, masterSystemTime=1675900465612 2023-02-08 23:54:25,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:25,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:25,630 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=0a59441954132eb44621ade02029d4dc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,631 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675900465630"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900465630"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900465630"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900465630"}]},"ts":"1675900465630"} 2023-02-08 23:54:25,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-08 23:54:25,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 0a59441954132eb44621ade02029d4dc, server=jenkins-hbase4.apache.org,36199,1675900441120 in 172 msec 2023-02-08 23:54:25,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-08 23:54:25,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0a59441954132eb44621ade02029d4dc, ASSIGN in 329 msec 2023-02-08 23:54:25,636 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:54:25,636 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900465636"}]},"ts":"1675900465636"} 2023-02-08 23:54:25,637 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-08 23:54:25,639 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:54:25,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 376 msec 2023-02-08 23:54:25,772 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:25,772 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:25,772 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:25,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,774 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:25,774 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:25,774 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 23:54:25,774 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 23:54:25,775 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,775 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,776 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:25,776 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:25,776 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:25,776 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:25,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:25,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:25,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:25,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:25,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:25,776 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,777 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:25,777 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:25,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:25,778 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:25,778 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:25,778 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. started... 2023-02-08 23:54:25,779 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0a59441954132eb44621ade02029d4dc 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 23:54:25,781 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,781 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,782 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,782 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:25,782 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,782 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:25,791 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/07e210660ce448e39e9b9091cc8c6a39 2023-02-08 23:54:25,803 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/3d18efc2303f4e3ea2ba3fc349726866 2023-02-08 23:54:25,807 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/07e210660ce448e39e9b9091cc8c6a39 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/07e210660ce448e39e9b9091cc8c6a39 2023-02-08 23:54:25,811 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/07e210660ce448e39e9b9091cc8c6a39, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:54:25,812 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/3d18efc2303f4e3ea2ba3fc349726866 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3d18efc2303f4e3ea2ba3fc349726866 2023-02-08 23:54:25,815 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3d18efc2303f4e3ea2ba3fc349726866, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:54:25,816 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 0a59441954132eb44621ade02029d4dc in 37ms, sequenceid=6, compaction requested=false 2023-02-08 23:54:25,816 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:25,817 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0a59441954132eb44621ade02029d4dc: 2023-02-08 23:54:25,817 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:25,817 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:25,817 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:25,817 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,817 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:25,817 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:25,818 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,818 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:25,818 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,818 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:25,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:25,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:25,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 23:54:25,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e955501[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:25,820 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:25,820 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,822 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,822 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,822 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,822 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:25,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,822 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,822 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:25,822 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:25,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,822 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:25,822 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:25,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,822 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:25,822 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:25,822 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:25,822 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:25,822 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:25,824 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:25,824 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:25,824 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:25,825 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:25,825 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:25,826 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:25,826 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:25,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:25,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:25,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:25,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:25,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:25,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:25,829 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 23:54:25,829 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:25,829 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:25,829 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,829 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:25,830 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,830 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,830 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,830 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:25,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:25,830 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:25,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,830 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:25,830 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:25,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:25,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:25,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:25,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:25,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:25,831 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:25,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:25,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:25,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,831 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:25,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:25,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,832 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:25,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:25,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:25,833 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:25,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,834 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:25,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:25,834 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:25,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:25,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:25,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:25,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:25,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:25,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:25,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:25,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:25,839 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:25,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:25,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:25,839 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:25,840 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:25,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:25,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:25,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:25,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:25,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:25,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:25,840 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:25,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:25,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:25,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:25,840 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,840 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:25,840 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,840 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:25,840 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:25,840 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:25,840 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,841 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,940 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:25,941 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:25,941 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:25,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:25,963 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:25,963 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:25,964 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:25,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:25,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,965 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:25,965 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:25,965 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 23:54:25,965 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 23:54:25,966 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,966 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,968 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:25,968 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:25,969 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:25,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:25,969 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:25,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:25,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:25,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:25,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:25,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,969 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:25,969 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:25,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:25,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:25,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:25,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,970 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:25,971 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:25,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:25,971 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:25,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:25,971 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:25,971 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:25,971 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. started... 2023-02-08 23:54:25,971 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0a59441954132eb44621ade02029d4dc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:25,972 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,973 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,973 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:25,973 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,973 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:25,973 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:25,988 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/662156a8d4454e3f9b39bd5706bea215 2023-02-08 23:54:26,034 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/f5c24e92b1274a82a4db8a9b84b51be9 2023-02-08 23:54:26,039 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/662156a8d4454e3f9b39bd5706bea215 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/662156a8d4454e3f9b39bd5706bea215 2023-02-08 23:54:26,044 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/662156a8d4454e3f9b39bd5706bea215, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:54:26,044 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/f5c24e92b1274a82a4db8a9b84b51be9 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f5c24e92b1274a82a4db8a9b84b51be9 2023-02-08 23:54:26,049 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f5c24e92b1274a82a4db8a9b84b51be9, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:54:26,050 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0a59441954132eb44621ade02029d4dc in 79ms, sequenceid=19, compaction requested=false 2023-02-08 23:54:26,050 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0a59441954132eb44621ade02029d4dc: 2023-02-08 23:54:26,050 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:26,050 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:26,050 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:26,050 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,050 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,050 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,052 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,052 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,052 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,052 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 23:54:26,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7faa3c2c[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:26,054 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:26,054 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,057 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,057 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,057 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,057 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,057 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,057 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,057 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:26,057 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,057 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,057 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:26,057 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,057 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,057 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:26,057 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:26,059 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,059 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,059 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,059 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,059 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,059 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,059 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,059 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,059 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,059 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:26,062 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:26,062 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 23:54:26,063 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:26,063 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:26,063 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,063 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:26,064 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,064 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,064 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,064 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,064 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,064 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,065 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,065 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,065 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,066 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,068 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,068 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,068 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,072 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,072 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:26,072 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:26,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,073 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:26,073 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:26,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,073 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,074 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,173 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:26,173 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:26,173 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:26,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:26,186 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:26,187 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:26,187 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:26,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:26,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,189 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:26,189 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,189 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 23:54:26,189 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 23:54:26,189 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,189 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,191 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,191 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,191 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,191 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,191 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,191 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:26,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,192 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,192 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,192 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,192 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,193 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,193 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,193 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,193 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,193 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:26,193 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:26,193 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:26,193 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. started... 2023-02-08 23:54:26,193 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0a59441954132eb44621ade02029d4dc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:26,195 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,195 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,195 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,195 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,195 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,195 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,203 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/7f17d7b5808247ecbb447fefd04dd77d 2023-02-08 23:54:26,214 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/be1ff474bd5c426faa85b294d8836664 2023-02-08 23:54:26,218 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/7f17d7b5808247ecbb447fefd04dd77d as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/7f17d7b5808247ecbb447fefd04dd77d 2023-02-08 23:54:26,221 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/7f17d7b5808247ecbb447fefd04dd77d, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:54:26,222 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/be1ff474bd5c426faa85b294d8836664 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/be1ff474bd5c426faa85b294d8836664 2023-02-08 23:54:26,225 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/be1ff474bd5c426faa85b294d8836664, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:54:26,226 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0a59441954132eb44621ade02029d4dc in 33ms, sequenceid=32, compaction requested=true 2023-02-08 23:54:26,226 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0a59441954132eb44621ade02029d4dc: 2023-02-08 23:54:26,226 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:26,226 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:26,227 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:26,227 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,227 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,227 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,232 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,232 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,232 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,232 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 23:54:26,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@588205a7[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:26,234 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:26,234 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,235 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,235 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,235 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,235 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,235 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,235 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,235 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,235 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,235 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:26,235 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:26,235 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,235 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,235 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,235 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:26,237 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,238 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,238 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:26,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:26,240 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 23:54:26,240 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:26,240 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:26,240 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,240 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:26,242 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,242 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,242 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,242 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,242 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,243 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,243 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,243 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,243 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,244 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,245 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,246 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,246 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,248 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,248 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,248 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,248 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,248 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,248 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:26,248 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:26,248 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,248 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,248 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,249 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,249 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:26,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,249 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:26,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,249 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,249 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,249 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,249 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,249 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,249 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,249 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,249 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,249 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,349 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:26,349 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:26,349 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:26,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:26,362 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:26,362 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:26,363 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:26,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:26,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,365 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:26,365 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,365 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 23:54:26,365 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 23:54:26,365 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,365 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,367 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,367 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,367 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,367 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,368 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,368 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,369 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,369 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,369 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,369 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,370 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:26,370 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:26,370 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:26,370 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. started... 2023-02-08 23:54:26,370 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0a59441954132eb44621ade02029d4dc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:26,371 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,371 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,371 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,371 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,371 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,371 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,380 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/9f37c71614b644028c60a208ec2dae92 2023-02-08 23:54:26,392 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/e9ed93d9f2cb4510856a03cfa143dbba 2023-02-08 23:54:26,397 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/9f37c71614b644028c60a208ec2dae92 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/9f37c71614b644028c60a208ec2dae92 2023-02-08 23:54:26,401 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/9f37c71614b644028c60a208ec2dae92, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:54:26,401 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/e9ed93d9f2cb4510856a03cfa143dbba as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/e9ed93d9f2cb4510856a03cfa143dbba 2023-02-08 23:54:26,405 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/e9ed93d9f2cb4510856a03cfa143dbba, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:54:26,406 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0a59441954132eb44621ade02029d4dc in 36ms, sequenceid=45, compaction requested=true 2023-02-08 23:54:26,406 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0a59441954132eb44621ade02029d4dc: 2023-02-08 23:54:26,406 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:26,406 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:26,406 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:26,406 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,406 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,406 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,407 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,407 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,408 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,408 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,410 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 23:54:26,410 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a1e190f[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:26,410 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:26,410 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,411 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,411 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,411 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,411 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,411 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,411 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,411 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,411 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,411 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:26,411 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,411 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,412 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:26,411 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:26,411 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:26,414 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,414 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,414 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,414 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,415 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,415 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,415 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,415 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,415 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,415 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:26,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:26,418 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 23:54:26,418 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:26,418 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:26,418 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,418 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:26,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,423 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,422 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,423 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,423 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,423 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,423 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,424 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,425 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,426 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,426 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,429 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,429 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,429 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,429 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,429 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:26,429 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:26,429 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,429 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,429 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,429 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,430 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,430 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,430 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,430 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,430 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:26,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,430 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,430 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:26,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,430 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,431 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,431 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,431 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,431 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,531 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:26,531 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:26,531 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:26,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:26,543 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:26,543 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:26,544 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:26,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:26,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,545 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:26,545 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,545 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 23:54:26,545 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 23:54:26,546 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,546 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,547 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,547 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,547 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,547 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,548 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,548 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,548 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:26,549 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:26,549 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. started... 2023-02-08 23:54:26,550 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0a59441954132eb44621ade02029d4dc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:26,551 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,551 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,551 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,552 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,552 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,552 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,558 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/47c0e414716643c0915dd58669faf777 2023-02-08 23:54:26,570 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/d42ed95ed42045a3a02129eb8f151fa2 2023-02-08 23:54:26,575 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/47c0e414716643c0915dd58669faf777 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/47c0e414716643c0915dd58669faf777 2023-02-08 23:54:26,579 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/47c0e414716643c0915dd58669faf777, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:54:26,580 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/d42ed95ed42045a3a02129eb8f151fa2 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d42ed95ed42045a3a02129eb8f151fa2 2023-02-08 23:54:26,584 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d42ed95ed42045a3a02129eb8f151fa2, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:54:26,585 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0a59441954132eb44621ade02029d4dc in 36ms, sequenceid=58, compaction requested=true 2023-02-08 23:54:26,586 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0a59441954132eb44621ade02029d4dc: 2023-02-08 23:54:26,586 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:26,586 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:26,586 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:26,586 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,586 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,586 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,588 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,588 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,588 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,589 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,591 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 23:54:26,591 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e576a10[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:26,591 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:26,591 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,592 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,592 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,592 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,592 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,592 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,592 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:26,592 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,593 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:26,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,593 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,593 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,593 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,593 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,593 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:26,593 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:26,594 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,594 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,594 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,595 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,595 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,595 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,597 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,597 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,597 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,597 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:26,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:26,601 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 23:54:26,601 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:26,601 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:26,601 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,601 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:26,602 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,602 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,602 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,602 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,602 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,602 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,602 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,603 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,603 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,604 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,606 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,606 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,606 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,609 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,609 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,609 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,609 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,609 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,609 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,609 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,609 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:26,610 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:26,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,609 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,609 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,609 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,609 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,610 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:26,610 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:26,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,610 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,610 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,610 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,610 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,610 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,610 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,611 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,710 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:26,710 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:26,711 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:26,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:26,723 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:26,723 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:26,724 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:26,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:26,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,726 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:26,726 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,726 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 23:54:26,726 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 23:54:26,726 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,726 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,734 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,735 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,734 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,734 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,735 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,735 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:26,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,736 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,736 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,736 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,736 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,736 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:26,737 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:26,737 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. started... 2023-02-08 23:54:26,737 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0a59441954132eb44621ade02029d4dc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:54:26,738 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,738 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,738 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,738 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,738 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,739 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,746 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/b1cd1edaaa404cc6bc0ec1a597a04c22 2023-02-08 23:54:26,759 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/ad0267ee4b044ef1a66b9bcf4a157841 2023-02-08 23:54:26,762 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/b1cd1edaaa404cc6bc0ec1a597a04c22 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/b1cd1edaaa404cc6bc0ec1a597a04c22 2023-02-08 23:54:26,766 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/b1cd1edaaa404cc6bc0ec1a597a04c22, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:54:26,767 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/ad0267ee4b044ef1a66b9bcf4a157841 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/ad0267ee4b044ef1a66b9bcf4a157841 2023-02-08 23:54:26,770 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/ad0267ee4b044ef1a66b9bcf4a157841, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:54:26,771 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0a59441954132eb44621ade02029d4dc in 34ms, sequenceid=71, compaction requested=true 2023-02-08 23:54:26,771 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0a59441954132eb44621ade02029d4dc: 2023-02-08 23:54:26,771 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:26,771 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:26,771 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:26,771 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,771 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,771 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,773 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,773 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,773 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,773 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,775 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 23:54:26,775 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40b260aa[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:26,775 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:26,775 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,776 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,776 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,776 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,776 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,776 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,776 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,776 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,777 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:26,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,776 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:26,777 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,777 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,777 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:26,776 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,777 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:26,779 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,779 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,779 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,780 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,780 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,780 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,780 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,780 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,780 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,780 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:26,784 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:26,784 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 23:54:26,784 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:26,784 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:26,784 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,784 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:26,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,786 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,786 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,786 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,787 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,788 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,788 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,789 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,791 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,791 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:26,791 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:26,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,791 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,792 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:26,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,792 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,792 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:26,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,792 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,792 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,792 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,792 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,892 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:26,892 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:26,892 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:26,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:26,904 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:26,904 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:26,905 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:26,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:26,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,906 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:26,907 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,907 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 23:54:26,907 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 23:54:26,907 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,907 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,910 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,910 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,910 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,910 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,910 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,910 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,911 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,911 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,911 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,911 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,911 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,912 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,912 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:26,912 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. started... 2023-02-08 23:54:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,912 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0a59441954132eb44621ade02029d4dc 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-08 23:54:26,913 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,914 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,914 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,914 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,915 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,915 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,921 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/a167a9ec1531417b957c21005dc49753 2023-02-08 23:54:26,925 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a167a9ec1531417b957c21005dc49753 2023-02-08 23:54:26,932 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90 2023-02-08 23:54:26,936 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e9f007c94ea4bcc998e5ee8e1e51e90 2023-02-08 23:54:26,936 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/a167a9ec1531417b957c21005dc49753 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/a167a9ec1531417b957c21005dc49753 2023-02-08 23:54:26,940 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a167a9ec1531417b957c21005dc49753 2023-02-08 23:54:26,940 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/a167a9ec1531417b957c21005dc49753, entries=9, sequenceid=83, filesize=5.1 K 2023-02-08 23:54:26,940 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90 2023-02-08 23:54:26,944 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e9f007c94ea4bcc998e5ee8e1e51e90 2023-02-08 23:54:26,944 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90, entries=9, sequenceid=83, filesize=5.1 K 2023-02-08 23:54:26,944 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 0a59441954132eb44621ade02029d4dc in 32ms, sequenceid=83, compaction requested=true 2023-02-08 23:54:26,945 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0a59441954132eb44621ade02029d4dc: 2023-02-08 23:54:26,945 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:26,945 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:26,945 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:26,945 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,945 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:26,945 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:26,946 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,946 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,947 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,947 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 23:54:26,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8d5fa26[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:26,948 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:26,949 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,950 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,950 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,950 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,951 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,951 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,951 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:26,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,951 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,951 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,951 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,951 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:26,951 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,951 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:26,951 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:26,951 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:26,959 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,959 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,959 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,959 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,959 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:26,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,959 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,959 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,960 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,960 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,960 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:26,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:26,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:26,963 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 23:54:26,963 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:26,963 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:26,963 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,963 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:26,964 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,964 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,964 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,964 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,964 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,964 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,964 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,965 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,965 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,966 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,967 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,967 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,967 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,970 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,970 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,970 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,970 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,970 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,970 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,970 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:26,970 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:26,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,970 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,970 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,970 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,970 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,970 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:26,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,970 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:26,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,970 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,971 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,971 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:26,971 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,971 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:26,971 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:26,971 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:26,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,971 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,071 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:27,071 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:27,071 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:27,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:27,079 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:27,079 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:27,080 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:27,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:27,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,081 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:27,082 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:27,082 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 23:54:27,082 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 23:54:27,082 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,082 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,084 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,084 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,084 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,084 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,085 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,085 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,085 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:27,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:27,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:27,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:27,086 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:27,086 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. started... 2023-02-08 23:54:27,086 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0a59441954132eb44621ade02029d4dc 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-08 23:54:27,087 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,088 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,088 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,088 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:27,088 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,088 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:27,095 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/1d802083a3c24e729fd7cc4d28b65e67 2023-02-08 23:54:27,099 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1d802083a3c24e729fd7cc4d28b65e67 2023-02-08 23:54:27,106 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/d5d5d0aa7c004461a1b53564f6b76bea 2023-02-08 23:54:27,110 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5d5d0aa7c004461a1b53564f6b76bea 2023-02-08 23:54:27,110 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/1d802083a3c24e729fd7cc4d28b65e67 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/1d802083a3c24e729fd7cc4d28b65e67 2023-02-08 23:54:27,114 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1d802083a3c24e729fd7cc4d28b65e67 2023-02-08 23:54:27,115 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/1d802083a3c24e729fd7cc4d28b65e67, entries=5, sequenceid=91, filesize=5.0 K 2023-02-08 23:54:27,115 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/d5d5d0aa7c004461a1b53564f6b76bea as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d5d5d0aa7c004461a1b53564f6b76bea 2023-02-08 23:54:27,121 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5d5d0aa7c004461a1b53564f6b76bea 2023-02-08 23:54:27,122 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d5d5d0aa7c004461a1b53564f6b76bea, entries=5, sequenceid=91, filesize=5.0 K 2023-02-08 23:54:27,123 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 0a59441954132eb44621ade02029d4dc in 36ms, sequenceid=91, compaction requested=true 2023-02-08 23:54:27,123 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0a59441954132eb44621ade02029d4dc: 2023-02-08 23:54:27,123 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:27,123 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:27,123 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:27,123 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,123 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:27,123 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:27,124 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,124 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:27,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:27,125 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,125 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 23:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62695c52[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:27,127 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:27,127 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,128 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,128 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,128 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:27,128 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,128 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:27,130 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:27,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:27,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:27,134 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 23:54:27,134 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:27,134 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:27,134 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,134 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:27,136 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,136 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,136 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:27,136 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,136 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:27,136 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,136 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,137 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,137 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,138 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,139 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,140 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,140 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,144 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,145 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,145 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,145 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,145 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:27,145 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:27,145 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,145 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,145 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:27,145 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,145 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,145 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,145 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:27,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,145 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:27,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,145 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,146 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,146 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,146 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,146 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,146 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,146 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,146 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,245 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:27,246 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:27,246 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:27,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:27,255 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:27,255 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:27,255 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:27,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:27,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,257 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:27,257 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:27,257 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 23:54:27,257 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 23:54:27,257 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,257 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,259 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,259 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,259 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,259 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,259 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,259 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:27,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:27,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:27,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,260 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,260 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:27,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,261 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,261 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:27,261 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:27,261 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,261 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,261 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:27,261 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 23:54:27,262 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:27,262 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:27,262 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:27,262 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. started... 2023-02-08 23:54:27,262 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0a59441954132eb44621ade02029d4dc 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-08 23:54:27,264 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,265 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,265 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:27,265 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,266 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,266 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:27,271 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/ebe3c354544a455ea14ab74024a1a9b1 2023-02-08 23:54:27,274 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ebe3c354544a455ea14ab74024a1a9b1 2023-02-08 23:54:27,283 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/3fe42b5e5f5f452abeeb859d3a43b350 2023-02-08 23:54:27,286 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3fe42b5e5f5f452abeeb859d3a43b350 2023-02-08 23:54:27,287 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/ebe3c354544a455ea14ab74024a1a9b1 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/ebe3c354544a455ea14ab74024a1a9b1 2023-02-08 23:54:27,291 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ebe3c354544a455ea14ab74024a1a9b1 2023-02-08 23:54:27,291 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/ebe3c354544a455ea14ab74024a1a9b1, entries=5, sequenceid=99, filesize=5.0 K 2023-02-08 23:54:27,292 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/3fe42b5e5f5f452abeeb859d3a43b350 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3fe42b5e5f5f452abeeb859d3a43b350 2023-02-08 23:54:27,295 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3fe42b5e5f5f452abeeb859d3a43b350 2023-02-08 23:54:27,296 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3fe42b5e5f5f452abeeb859d3a43b350, entries=5, sequenceid=99, filesize=5.0 K 2023-02-08 23:54:27,296 INFO [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 0a59441954132eb44621ade02029d4dc in 34ms, sequenceid=99, compaction requested=true 2023-02-08 23:54:27,296 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0a59441954132eb44621ade02029d4dc: 2023-02-08 23:54:27,296 DEBUG [rs(jenkins-hbase4.apache.org,36199,1675900441120)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:27,296 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:27,297 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:27,297 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,297 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 23:54:27,297 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 23:54:27,298 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,298 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:27,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:27,298 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,298 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:27,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:27,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:27,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:27,300 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 23:54:27,300 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@373bade5[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:27,300 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 23:54:27,300 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,301 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,301 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,302 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,302 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:27,302 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,302 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:27,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,302 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:27,302 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 23:54:27,302 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:27,302 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:27,302 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:54:27,302 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 23:54:27,302 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:54:27,302 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:54:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:27,305 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:27,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 23:54:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:27,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:27,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 23:54:27,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:27,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:27,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:27,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36199,1675900441120': 2023-02-08 23:54:27,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:54:27,308 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 23:54:27,308 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:27,308 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:27,308 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,308 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:27,310 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,310 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,310 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,310 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,310 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,310 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,310 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:27,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,310 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,311 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,311 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:27,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,312 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,313 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:27,313 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,315 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,315 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,315 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,315 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,316 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,316 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,316 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 23:54:27,316 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:27,316 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,316 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,316 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,316 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,316 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,316 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:54:27,316 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:27,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,316 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,316 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,316 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,317 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:27,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,317 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:54:27,317 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,317 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:27,416 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:27,417 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:27,417 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:27,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:27,420 DEBUG [Listener at localhost/37973] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,420 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:27,422 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:27,422 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:27,422 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:27,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0a59441954132eb44621ade02029d4dc, UNASSIGN}] 2023-02-08 23:54:27,424 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0a59441954132eb44621ade02029d4dc, UNASSIGN 2023-02-08 23:54:27,424 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=0a59441954132eb44621ade02029d4dc, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:27,425 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675900467424"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900467424"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900467424"}]},"ts":"1675900467424"} 2023-02-08 23:54:27,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 0a59441954132eb44621ade02029d4dc, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:27,578 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,578 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0a59441954132eb44621ade02029d4dc, disabling compactions & flushes 2023-02-08 23:54:27,578 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:27,578 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:27,578 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. after waiting 0 ms 2023-02-08 23:54:27,578 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:27,578 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 0a59441954132eb44621ade02029d4dc 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-08 23:54:27,588 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/25827684de8d44188b27dfe977104e72 2023-02-08 23:54:27,592 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25827684de8d44188b27dfe977104e72 2023-02-08 23:54:27,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/f538e988a65f4cfe95e8c5898426419d 2023-02-08 23:54:27,604 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f538e988a65f4cfe95e8c5898426419d 2023-02-08 23:54:27,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf1/25827684de8d44188b27dfe977104e72 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/25827684de8d44188b27dfe977104e72 2023-02-08 23:54:27,608 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25827684de8d44188b27dfe977104e72 2023-02-08 23:54:27,608 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/25827684de8d44188b27dfe977104e72, entries=1, sequenceid=103, filesize=4.9 K 2023-02-08 23:54:27,609 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.tmp/cf2/f538e988a65f4cfe95e8c5898426419d as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f538e988a65f4cfe95e8c5898426419d 2023-02-08 23:54:27,612 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f538e988a65f4cfe95e8c5898426419d 2023-02-08 23:54:27,612 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f538e988a65f4cfe95e8c5898426419d, entries=1, sequenceid=103, filesize=4.9 K 2023-02-08 23:54:27,613 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 0a59441954132eb44621ade02029d4dc in 35ms, sequenceid=103, compaction requested=true 2023-02-08 23:54:27,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-08 23:54:27,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:27,618 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. 2023-02-08 23:54:27,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0a59441954132eb44621ade02029d4dc: Waiting for close lock at 1675900467578Running coprocessor pre-close hooks at 1675900467578Disabling compacts and flushes for region at 1675900467578Disabling writes for close at 1675900467578Obtaining lock to block concurrent updates at 1675900467578Preparing flush snapshotting stores in 0a59441954132eb44621ade02029d4dc at 1675900467578Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1675900467578Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc. at 1675900467579 (+1 ms)Flushing 0a59441954132eb44621ade02029d4dc/cf1: creating writer at 1675900467580 (+1 ms)Flushing 0a59441954132eb44621ade02029d4dc/cf1: appending metadata at 1675900467582 (+2 ms)Flushing 0a59441954132eb44621ade02029d4dc/cf1: closing flushed file at 1675900467582Flushing 0a59441954132eb44621ade02029d4dc/cf2: creating writer at 1675900467592 (+10 ms)Flushing 0a59441954132eb44621ade02029d4dc/cf2: appending metadata at 1675900467594 (+2 ms)Flushing 0a59441954132eb44621ade02029d4dc/cf2: closing flushed file at 1675900467594Flushing 0a59441954132eb44621ade02029d4dc/cf1: reopening flushed file at 1675900467605 (+11 ms)Flushing 0a59441954132eb44621ade02029d4dc/cf2: reopening flushed file at 1675900467609 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 0a59441954132eb44621ade02029d4dc in 35ms, sequenceid=103, compaction requested=true at 1675900467613 (+4 ms)Writing region close event to WAL at 1675900467615 (+2 ms)Running coprocessor post-close hooks at 1675900467618 (+3 ms)Closed at 1675900467618 2023-02-08 23:54:27,620 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,620 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=0a59441954132eb44621ade02029d4dc, regionState=CLOSED 2023-02-08 23:54:27,620 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675900467620"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900467620"}]},"ts":"1675900467620"} 2023-02-08 23:54:27,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-08 23:54:27,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 0a59441954132eb44621ade02029d4dc, server=jenkins-hbase4.apache.org,36199,1675900441120 in 195 msec 2023-02-08 23:54:27,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-08 23:54:27,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0a59441954132eb44621ade02029d4dc, UNASSIGN in 200 msec 2023-02-08 23:54:27,636 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=0a59441954132eb44621ade02029d4dc, threads=16 2023-02-08 23:54:27,637 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3d18efc2303f4e3ea2ba3fc349726866 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,637 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3fe42b5e5f5f452abeeb859d3a43b350 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,637 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,640 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/ad0267ee4b044ef1a66b9bcf4a157841 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,644 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/be1ff474bd5c426faa85b294d8836664 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,644 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d42ed95ed42045a3a02129eb8f151fa2 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,644 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d5d5d0aa7c004461a1b53564f6b76bea for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,644 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/e9ed93d9f2cb4510856a03cfa143dbba for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,644 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f5c24e92b1274a82a4db8a9b84b51be9 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,644 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/07e210660ce448e39e9b9091cc8c6a39 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,645 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/1d802083a3c24e729fd7cc4d28b65e67 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,645 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f538e988a65f4cfe95e8c5898426419d for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,645 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/25827684de8d44188b27dfe977104e72 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,647 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/47c0e414716643c0915dd58669faf777 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,648 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/662156a8d4454e3f9b39bd5706bea215 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,652 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/7f17d7b5808247ecbb447fefd04dd77d for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,655 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e9f007c94ea4bcc998e5ee8e1e51e90 2023-02-08 23:54:27,659 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/e9ed93d9f2cb4510856a03cfa143dbba, top=false 2023-02-08 23:54:27,659 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f538e988a65f4cfe95e8c5898426419d 2023-02-08 23:54:27,659 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/ad0267ee4b044ef1a66b9bcf4a157841, top=false 2023-02-08 23:54:27,661 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3fe42b5e5f5f452abeeb859d3a43b350 2023-02-08 23:54:27,662 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d42ed95ed42045a3a02129eb8f151fa2, top=false 2023-02-08 23:54:27,667 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25827684de8d44188b27dfe977104e72 2023-02-08 23:54:27,673 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1d802083a3c24e729fd7cc4d28b65e67 2023-02-08 23:54:27,680 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1d802083a3c24e729fd7cc4d28b65e67 2023-02-08 23:54:27,680 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/1d802083a3c24e729fd7cc4d28b65e67, top=true 2023-02-08 23:54:27,680 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f538e988a65f4cfe95e8c5898426419d 2023-02-08 23:54:27,680 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f538e988a65f4cfe95e8c5898426419d, top=true 2023-02-08 23:54:27,700 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5d5d0aa7c004461a1b53564f6b76bea 2023-02-08 23:54:27,702 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3fe42b5e5f5f452abeeb859d3a43b350 2023-02-08 23:54:27,702 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3fe42b5e5f5f452abeeb859d3a43b350, top=true 2023-02-08 23:54:27,702 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25827684de8d44188b27dfe977104e72 2023-02-08 23:54:27,702 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/25827684de8d44188b27dfe977104e72, top=true 2023-02-08 23:54:27,702 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/47c0e414716643c0915dd58669faf777, top=false 2023-02-08 23:54:27,708 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5d5d0aa7c004461a1b53564f6b76bea 2023-02-08 23:54:27,708 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d5d5d0aa7c004461a1b53564f6b76bea, top=true 2023-02-08 23:54:27,720 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.splits/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-f538e988a65f4cfe95e8c5898426419d for child: 1f5b68527c02157d9f50f31363d383eb, parent: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,720 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f538e988a65f4cfe95e8c5898426419d for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,720 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/9f37c71614b644028c60a208ec2dae92 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,720 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.splits/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-1d802083a3c24e729fd7cc4d28b65e67 for child: 1f5b68527c02157d9f50f31363d383eb, parent: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,720 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/1d802083a3c24e729fd7cc4d28b65e67 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,721 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/a167a9ec1531417b957c21005dc49753 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,725 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/9f37c71614b644028c60a208ec2dae92, top=false 2023-02-08 23:54:27,727 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a167a9ec1531417b957c21005dc49753 2023-02-08 23:54:27,736 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.splits/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-e9ed93d9f2cb4510856a03cfa143dbba for child: f91831a12657ef5da469c59d1a700126, parent: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,736 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.splits/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ad0267ee4b044ef1a66b9bcf4a157841 for child: f91831a12657ef5da469c59d1a700126, parent: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,738 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.splits/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-25827684de8d44188b27dfe977104e72 for child: 1f5b68527c02157d9f50f31363d383eb, parent: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,738 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/25827684de8d44188b27dfe977104e72 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,738 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/b1cd1edaaa404cc6bc0ec1a597a04c22 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,738 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.splits/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-47c0e414716643c0915dd58669faf777 for child: f91831a12657ef5da469c59d1a700126, parent: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,739 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.splits/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-3fe42b5e5f5f452abeeb859d3a43b350 for child: 1f5b68527c02157d9f50f31363d383eb, parent: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,739 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3fe42b5e5f5f452abeeb859d3a43b350 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,739 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/ebe3c354544a455ea14ab74024a1a9b1 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,739 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/ad0267ee4b044ef1a66b9bcf4a157841 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,741 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/e9ed93d9f2cb4510856a03cfa143dbba for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,742 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/47c0e414716643c0915dd58669faf777 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,743 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ebe3c354544a455ea14ab74024a1a9b1 2023-02-08 23:54:27,744 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/b1cd1edaaa404cc6bc0ec1a597a04c22, top=false 2023-02-08 23:54:27,746 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ebe3c354544a455ea14ab74024a1a9b1 2023-02-08 23:54:27,746 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/ebe3c354544a455ea14ab74024a1a9b1, top=true 2023-02-08 23:54:27,762 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.splits/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d42ed95ed42045a3a02129eb8f151fa2 for child: f91831a12657ef5da469c59d1a700126, parent: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,767 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d42ed95ed42045a3a02129eb8f151fa2 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,767 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e9f007c94ea4bcc998e5ee8e1e51e90 2023-02-08 23:54:27,776 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.splits/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d5d5d0aa7c004461a1b53564f6b76bea for child: 1f5b68527c02157d9f50f31363d383eb, parent: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,776 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d5d5d0aa7c004461a1b53564f6b76bea for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,781 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.splits/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-9f37c71614b644028c60a208ec2dae92 for child: f91831a12657ef5da469c59d1a700126, parent: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,784 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/9f37c71614b644028c60a208ec2dae92 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,790 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.splits/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-b1cd1edaaa404cc6bc0ec1a597a04c22 for child: f91831a12657ef5da469c59d1a700126, parent: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,790 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/.splits/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ebe3c354544a455ea14ab74024a1a9b1 for child: 1f5b68527c02157d9f50f31363d383eb, parent: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,790 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/ebe3c354544a455ea14ab74024a1a9b1 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,795 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/b1cd1edaaa404cc6bc0ec1a597a04c22 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,804 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3d18efc2303f4e3ea2ba3fc349726866 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,809 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a167a9ec1531417b957c21005dc49753 2023-02-08 23:54:27,810 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/7f17d7b5808247ecbb447fefd04dd77d for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,816 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,820 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/be1ff474bd5c426faa85b294d8836664 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,826 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f5c24e92b1274a82a4db8a9b84b51be9 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,832 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/07e210660ce448e39e9b9091cc8c6a39 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,843 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/a167a9ec1531417b957c21005dc49753 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,843 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/662156a8d4454e3f9b39bd5706bea215 for region: 0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:27,843 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 0a59441954132eb44621ade02029d4dc Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 23:54:27,882 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-08 23:54:27,884 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-08 23:54:27,886 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675900467886"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1675900467886"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1675900467886"}]},"ts":"1675900467886"} 2023-02-08 23:54:27,886 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675900467886"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900467886"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900467886"}]},"ts":"1675900467886"} 2023-02-08 23:54:27,886 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675900467886"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900467886"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900467886"}]},"ts":"1675900467886"} 2023-02-08 23:54:27,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=f91831a12657ef5da469c59d1a700126, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f5b68527c02157d9f50f31363d383eb, ASSIGN}] 2023-02-08 23:54:27,890 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f5b68527c02157d9f50f31363d383eb, ASSIGN 2023-02-08 23:54:27,891 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f5b68527c02157d9f50f31363d383eb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=false 2023-02-08 23:54:27,892 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=f91831a12657ef5da469c59d1a700126, ASSIGN 2023-02-08 23:54:27,892 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=f91831a12657ef5da469c59d1a700126, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=false 2023-02-08 23:54:28,041 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 23:54:28,043 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=f91831a12657ef5da469c59d1a700126, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:28,043 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675900468043"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900468043"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900468043"}]},"ts":"1675900468043"} 2023-02-08 23:54:28,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure f91831a12657ef5da469c59d1a700126, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:28,047 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=1f5b68527c02157d9f50f31363d383eb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:28,047 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675900468047"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900468047"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900468047"}]},"ts":"1675900468047"} 2023-02-08 23:54:28,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 1f5b68527c02157d9f50f31363d383eb, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:28,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1f5b68527c02157d9f50f31363d383eb, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.', STARTKEY => '60', ENDKEY => ''} 2023-02-08 23:54:28,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:28,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:28,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:28,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:28,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:28,201 INFO [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:28,202 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1 2023-02-08 23:54:28,202 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1 2023-02-08 23:54:28,202 INFO [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1f5b68527c02157d9f50f31363d383eb columnFamilyName cf1 2023-02-08 23:54:28,211 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/07e210660ce448e39e9b9091cc8c6a39.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/07e210660ce448e39e9b9091cc8c6a39-top 2023-02-08 23:54:28,215 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/662156a8d4454e3f9b39bd5706bea215.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/662156a8d4454e3f9b39bd5706bea215-top 2023-02-08 23:54:28,219 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/7f17d7b5808247ecbb447fefd04dd77d.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/7f17d7b5808247ecbb447fefd04dd77d-top 2023-02-08 23:54:28,222 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,223 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/a167a9ec1531417b957c21005dc49753-top 2023-02-08 23:54:28,226 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1d802083a3c24e729fd7cc4d28b65e67 2023-02-08 23:54:28,227 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-1d802083a3c24e729fd7cc4d28b65e67 2023-02-08 23:54:28,230 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25827684de8d44188b27dfe977104e72 2023-02-08 23:54:28,230 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-25827684de8d44188b27dfe977104e72 2023-02-08 23:54:28,234 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ebe3c354544a455ea14ab74024a1a9b1 2023-02-08 23:54:28,234 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ebe3c354544a455ea14ab74024a1a9b1 2023-02-08 23:54:28,234 INFO [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(310): Store=1f5b68527c02157d9f50f31363d383eb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:28,234 INFO [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:28,235 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2 2023-02-08 23:54:28,235 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2 2023-02-08 23:54:28,236 INFO [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1f5b68527c02157d9f50f31363d383eb columnFamilyName cf2 2023-02-08 23:54:28,248 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/3d18efc2303f4e3ea2ba3fc349726866.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3d18efc2303f4e3ea2ba3fc349726866-top 2023-02-08 23:54:28,250 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,251 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90-top 2023-02-08 23:54:28,254 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/be1ff474bd5c426faa85b294d8836664.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/be1ff474bd5c426faa85b294d8836664-top 2023-02-08 23:54:28,257 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/f5c24e92b1274a82a4db8a9b84b51be9.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f5c24e92b1274a82a4db8a9b84b51be9-top 2023-02-08 23:54:28,259 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3fe42b5e5f5f452abeeb859d3a43b350 2023-02-08 23:54:28,260 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-3fe42b5e5f5f452abeeb859d3a43b350 2023-02-08 23:54:28,262 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5d5d0aa7c004461a1b53564f6b76bea 2023-02-08 23:54:28,262 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d5d5d0aa7c004461a1b53564f6b76bea 2023-02-08 23:54:28,265 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f538e988a65f4cfe95e8c5898426419d 2023-02-08 23:54:28,265 DEBUG [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-f538e988a65f4cfe95e8c5898426419d 2023-02-08 23:54:28,265 INFO [StoreOpener-1f5b68527c02157d9f50f31363d383eb-1] regionserver.HStore(310): Store=1f5b68527c02157d9f50f31363d383eb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:28,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:28,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:28,268 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:28,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:28,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1f5b68527c02157d9f50f31363d383eb; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61199466, jitterRate=-0.08805689215660095}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:28,270 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1f5b68527c02157d9f50f31363d383eb: Running coprocessor pre-open hook at 1675900468200Writing region info on filesystem at 1675900468200Initializing all the Stores at 1675900468201 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900468201Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900468201Cleaning up temporary data from old regions at 1675900468268 (+67 ms)Cleaning up detritus from prior splits at 1675900468268Running coprocessor post-open hooks at 1675900468270 (+2 ms)Region opened successfully at 1675900468270 2023-02-08 23:54:28,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb., pid=78, masterSystemTime=1675900468196 2023-02-08 23:54:28,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:28,272 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:54:28,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:28,272 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:54:28,272 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,272 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1912): 1f5b68527c02157d9f50f31363d383eb/cf1 is initiating minor compaction (all files) 2023-02-08 23:54:28,272 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,273 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1912): 1f5b68527c02157d9f50f31363d383eb/cf2 is initiating minor compaction (all files) 2023-02-08 23:54:28,273 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1f5b68527c02157d9f50f31363d383eb/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,273 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1f5b68527c02157d9f50f31363d383eb/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,273 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/07e210660ce448e39e9b9091cc8c6a39.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/07e210660ce448e39e9b9091cc8c6a39-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/662156a8d4454e3f9b39bd5706bea215.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/662156a8d4454e3f9b39bd5706bea215-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/7f17d7b5808247ecbb447fefd04dd77d.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/7f17d7b5808247ecbb447fefd04dd77d-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/a167a9ec1531417b957c21005dc49753-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-1d802083a3c24e729fd7cc4d28b65e67, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ebe3c354544a455ea14ab74024a1a9b1, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-25827684de8d44188b27dfe977104e72] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/.tmp, totalSize=34.8 K 2023-02-08 23:54:28,273 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/3d18efc2303f4e3ea2ba3fc349726866.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3d18efc2303f4e3ea2ba3fc349726866-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/f5c24e92b1274a82a4db8a9b84b51be9.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f5c24e92b1274a82a4db8a9b84b51be9-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/be1ff474bd5c426faa85b294d8836664.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/be1ff474bd5c426faa85b294d8836664-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d5d5d0aa7c004461a1b53564f6b76bea, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-3fe42b5e5f5f452abeeb859d3a43b350, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-f538e988a65f4cfe95e8c5898426419d] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/.tmp, totalSize=34.8 K 2023-02-08 23:54:28,273 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 07e210660ce448e39e9b9091cc8c6a39.0a59441954132eb44621ade02029d4dc, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675900465767 2023-02-08 23:54:28,273 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting 3d18efc2303f4e3ea2ba3fc349726866.0a59441954132eb44621ade02029d4dc, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675900465767 2023-02-08 23:54:28,274 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 662156a8d4454e3f9b39bd5706bea215.0a59441954132eb44621ade02029d4dc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675900465943 2023-02-08 23:54:28,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,274 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting f5c24e92b1274a82a4db8a9b84b51be9.0a59441954132eb44621ade02029d4dc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675900465943 2023-02-08 23:54:28,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,274 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 7f17d7b5808247ecbb447fefd04dd77d.0a59441954132eb44621ade02029d4dc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675900466175 2023-02-08 23:54:28,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f91831a12657ef5da469c59d1a700126, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.', STARTKEY => '', ENDKEY => '60'} 2023-02-08 23:54:28,274 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting be1ff474bd5c426faa85b294d8836664.0a59441954132eb44621ade02029d4dc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675900466175 2023-02-08 23:54:28,275 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=1f5b68527c02157d9f50f31363d383eb, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:28,275 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675900466894 2023-02-08 23:54:28,275 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675900468274"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900468274"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900468274"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900468274"}]},"ts":"1675900468274"} 2023-02-08 23:54:28,275 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting 4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675900466894 2023-02-08 23:54:28,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:28,275 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-1d802083a3c24e729fd7cc4d28b65e67, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-08 23:54:28,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter f91831a12657ef5da469c59d1a700126 2023-02-08 23:54:28,275 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d5d5d0aa7c004461a1b53564f6b76bea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-08 23:54:28,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:28,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f91831a12657ef5da469c59d1a700126 2023-02-08 23:54:28,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f91831a12657ef5da469c59d1a700126 2023-02-08 23:54:28,275 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ebe3c354544a455ea14ab74024a1a9b1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-08 23:54:28,276 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-3fe42b5e5f5f452abeeb859d3a43b350, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-08 23:54:28,276 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-f538e988a65f4cfe95e8c5898426419d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-08 23:54:28,276 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-25827684de8d44188b27dfe977104e72, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-08 23:54:28,278 INFO [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f91831a12657ef5da469c59d1a700126 2023-02-08 23:54:28,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-08 23:54:28,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 1f5b68527c02157d9f50f31363d383eb, server=jenkins-hbase4.apache.org,36199,1675900441120 in 228 msec 2023-02-08 23:54:28,279 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1 2023-02-08 23:54:28,279 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1 2023-02-08 23:54:28,280 INFO [StoreOpener-f91831a12657ef5da469c59d1a700126-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f91831a12657ef5da469c59d1a700126 columnFamilyName cf1 2023-02-08 23:54:28,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f5b68527c02157d9f50f31363d383eb, ASSIGN in 390 msec 2023-02-08 23:54:28,293 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/07e210660ce448e39e9b9091cc8c6a39.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/07e210660ce448e39e9b9091cc8c6a39-bottom 2023-02-08 23:54:28,296 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] throttle.PressureAwareThroughputController(145): 1f5b68527c02157d9f50f31363d383eb#cf2#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:28,298 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/662156a8d4454e3f9b39bd5706bea215.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/662156a8d4454e3f9b39bd5706bea215-bottom 2023-02-08 23:54:28,302 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/7f17d7b5808247ecbb447fefd04dd77d.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/7f17d7b5808247ecbb447fefd04dd77d-bottom 2023-02-08 23:54:28,307 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1f5b68527c02157d9f50f31363d383eb#cf1#compaction#73 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:28,311 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,311 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/a167a9ec1531417b957c21005dc49753-bottom 2023-02-08 23:54:28,321 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 753505c5d4d24c47b38f69b83a230c0d: NONE, but ROW specified in column family configuration 2023-02-08 23:54:28,321 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-47c0e414716643c0915dd58669faf777 2023-02-08 23:54:28,322 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/.tmp/cf2/753505c5d4d24c47b38f69b83a230c0d as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/753505c5d4d24c47b38f69b83a230c0d 2023-02-08 23:54:28,325 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-9f37c71614b644028c60a208ec2dae92 2023-02-08 23:54:28,327 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 753505c5d4d24c47b38f69b83a230c0d: NONE, but ROW specified in column family configuration 2023-02-08 23:54:28,328 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1f5b68527c02157d9f50f31363d383eb/cf2 of 1f5b68527c02157d9f50f31363d383eb into 753505c5d4d24c47b38f69b83a230c0d(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:28,328 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1f5b68527c02157d9f50f31363d383eb: 2023-02-08 23:54:28,328 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb., storeName=1f5b68527c02157d9f50f31363d383eb/cf2, priority=9, startTime=1675900468272; duration=0sec 2023-02-08 23:54:28,328 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:28,333 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for d9db015f2bc64bed9b2480c34e3ee9c2: NONE, but ROW specified in column family configuration 2023-02-08 23:54:28,334 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-b1cd1edaaa404cc6bc0ec1a597a04c22 2023-02-08 23:54:28,334 INFO [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(310): Store=f91831a12657ef5da469c59d1a700126/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:28,334 INFO [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f91831a12657ef5da469c59d1a700126 2023-02-08 23:54:28,334 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/.tmp/cf1/d9db015f2bc64bed9b2480c34e3ee9c2 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/d9db015f2bc64bed9b2480c34e3ee9c2 2023-02-08 23:54:28,335 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2 2023-02-08 23:54:28,335 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2 2023-02-08 23:54:28,335 INFO [StoreOpener-f91831a12657ef5da469c59d1a700126-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f91831a12657ef5da469c59d1a700126 columnFamilyName cf2 2023-02-08 23:54:28,338 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for d9db015f2bc64bed9b2480c34e3ee9c2: NONE, but ROW specified in column family configuration 2023-02-08 23:54:28,339 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1f5b68527c02157d9f50f31363d383eb/cf1 of 1f5b68527c02157d9f50f31363d383eb into d9db015f2bc64bed9b2480c34e3ee9c2(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:28,339 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1f5b68527c02157d9f50f31363d383eb: 2023-02-08 23:54:28,339 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb., storeName=1f5b68527c02157d9f50f31363d383eb/cf1, priority=9, startTime=1675900468271; duration=0sec 2023-02-08 23:54:28,340 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:28,345 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/3d18efc2303f4e3ea2ba3fc349726866.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3d18efc2303f4e3ea2ba3fc349726866-bottom 2023-02-08 23:54:28,348 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,348 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90-bottom 2023-02-08 23:54:28,351 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/be1ff474bd5c426faa85b294d8836664.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/be1ff474bd5c426faa85b294d8836664-bottom 2023-02-08 23:54:28,354 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/f5c24e92b1274a82a4db8a9b84b51be9.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f5c24e92b1274a82a4db8a9b84b51be9-bottom 2023-02-08 23:54:28,357 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ad0267ee4b044ef1a66b9bcf4a157841 2023-02-08 23:54:28,363 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d42ed95ed42045a3a02129eb8f151fa2 2023-02-08 23:54:28,367 DEBUG [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-e9ed93d9f2cb4510856a03cfa143dbba 2023-02-08 23:54:28,367 INFO [StoreOpener-f91831a12657ef5da469c59d1a700126-1] regionserver.HStore(310): Store=f91831a12657ef5da469c59d1a700126/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:28,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126 2023-02-08 23:54:28,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126 2023-02-08 23:54:28,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:28,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f91831a12657ef5da469c59d1a700126 2023-02-08 23:54:28,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f91831a12657ef5da469c59d1a700126; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71268453, jitterRate=0.06198270618915558}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:28,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f91831a12657ef5da469c59d1a700126: Running coprocessor pre-open hook at 1675900468276Writing region info on filesystem at 1675900468276Initializing all the Stores at 1675900468278 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900468278Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900468278Cleaning up temporary data from old regions at 1675900468369 (+91 ms)Cleaning up detritus from prior splits at 1675900468370 (+1 ms)Running coprocessor post-open hooks at 1675900468372 (+2 ms)Region opened successfully at 1675900468372 2023-02-08 23:54:28,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126., pid=77, masterSystemTime=1675900468196 2023-02-08 23:54:28,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:28,374 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:54:28,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:28,374 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:54:28,374 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,374 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1912): f91831a12657ef5da469c59d1a700126/cf1 is initiating minor compaction (all files) 2023-02-08 23:54:28,374 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,374 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1912): f91831a12657ef5da469c59d1a700126/cf2 is initiating minor compaction (all files) 2023-02-08 23:54:28,374 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f91831a12657ef5da469c59d1a700126/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,374 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f91831a12657ef5da469c59d1a700126/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,375 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/07e210660ce448e39e9b9091cc8c6a39.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/07e210660ce448e39e9b9091cc8c6a39-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/662156a8d4454e3f9b39bd5706bea215.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/662156a8d4454e3f9b39bd5706bea215-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/7f17d7b5808247ecbb447fefd04dd77d.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/7f17d7b5808247ecbb447fefd04dd77d-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-9f37c71614b644028c60a208ec2dae92, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-47c0e414716643c0915dd58669faf777, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-b1cd1edaaa404cc6bc0ec1a597a04c22, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/a167a9ec1531417b957c21005dc49753-bottom] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/.tmp, totalSize=35.0 K 2023-02-08 23:54:28,375 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/3d18efc2303f4e3ea2ba3fc349726866.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3d18efc2303f4e3ea2ba3fc349726866-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/f5c24e92b1274a82a4db8a9b84b51be9.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f5c24e92b1274a82a4db8a9b84b51be9-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/be1ff474bd5c426faa85b294d8836664.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/be1ff474bd5c426faa85b294d8836664-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-e9ed93d9f2cb4510856a03cfa143dbba, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d42ed95ed42045a3a02129eb8f151fa2, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ad0267ee4b044ef1a66b9bcf4a157841, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90-bottom] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/.tmp, totalSize=35.0 K 2023-02-08 23:54:28,375 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 07e210660ce448e39e9b9091cc8c6a39.0a59441954132eb44621ade02029d4dc, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900465767 2023-02-08 23:54:28,375 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting 3d18efc2303f4e3ea2ba3fc349726866.0a59441954132eb44621ade02029d4dc, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900465767 2023-02-08 23:54:28,376 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 662156a8d4454e3f9b39bd5706bea215.0a59441954132eb44621ade02029d4dc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900465943 2023-02-08 23:54:28,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,376 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting f5c24e92b1274a82a4db8a9b84b51be9.0a59441954132eb44621ade02029d4dc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900465943 2023-02-08 23:54:28,376 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 7f17d7b5808247ecbb447fefd04dd77d.0a59441954132eb44621ade02029d4dc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900466175 2023-02-08 23:54:28,376 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=f91831a12657ef5da469c59d1a700126, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:28,377 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675900468376"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900468376"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900468376"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900468376"}]},"ts":"1675900468376"} 2023-02-08 23:54:28,377 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting be1ff474bd5c426faa85b294d8836664.0a59441954132eb44621ade02029d4dc, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900466175 2023-02-08 23:54:28,377 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-9f37c71614b644028c60a208ec2dae92, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900466351 2023-02-08 23:54:28,377 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-e9ed93d9f2cb4510856a03cfa143dbba, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900466351 2023-02-08 23:54:28,377 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-47c0e414716643c0915dd58669faf777, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900466533 2023-02-08 23:54:28,378 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d42ed95ed42045a3a02129eb8f151fa2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900466533 2023-02-08 23:54:28,378 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-b1cd1edaaa404cc6bc0ec1a597a04c22, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900466712 2023-02-08 23:54:28,378 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ad0267ee4b044ef1a66b9bcf4a157841, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900466712 2023-02-08 23:54:28,378 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675900466894 2023-02-08 23:54:28,378 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting 4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675900466894 2023-02-08 23:54:28,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-08 23:54:28,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure f91831a12657ef5da469c59d1a700126, server=jenkins-hbase4.apache.org,36199,1675900441120 in 333 msec 2023-02-08 23:54:28,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-02-08 23:54:28,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=f91831a12657ef5da469c59d1a700126, ASSIGN in 492 msec 2023-02-08 23:54:28,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb in 963 msec 2023-02-08 23:54:28,395 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] throttle.PressureAwareThroughputController(145): f91831a12657ef5da469c59d1a700126#cf1#compaction#74 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:28,396 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] throttle.PressureAwareThroughputController(145): f91831a12657ef5da469c59d1a700126#cf2#compaction#75 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:28,417 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/.tmp/cf1/9aaa63f230a54ddc8a10bd9042049011 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/9aaa63f230a54ddc8a10bd9042049011 2023-02-08 23:54:28,417 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/.tmp/cf2/9bf35bc1c4e14a0ba16ae79db1893fe8 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/9bf35bc1c4e14a0ba16ae79db1893fe8 2023-02-08 23:54:28,421 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f91831a12657ef5da469c59d1a700126/cf1 of f91831a12657ef5da469c59d1a700126 into 9aaa63f230a54ddc8a10bd9042049011(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:28,421 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f91831a12657ef5da469c59d1a700126/cf2 of f91831a12657ef5da469c59d1a700126 into 9bf35bc1c4e14a0ba16ae79db1893fe8(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:28,421 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f91831a12657ef5da469c59d1a700126: 2023-02-08 23:54:28,422 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f91831a12657ef5da469c59d1a700126: 2023-02-08 23:54:28,422 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126., storeName=f91831a12657ef5da469c59d1a700126/cf1, priority=9, startTime=1675900468373; duration=0sec 2023-02-08 23:54:28,422 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126., storeName=f91831a12657ef5da469c59d1a700126/cf2, priority=9, startTime=1675900468374; duration=0sec 2023-02-08 23:54:28,422 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:28,422 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:28,431 DEBUG [Listener at localhost/37973] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 23:54:28,436 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44982, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 23:54:28,436 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36199] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,437 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36199] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 23:54:28,437 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36199] regionserver.HStore(1912): f91831a12657ef5da469c59d1a700126/cf1 is initiating major compaction (all files) 2023-02-08 23:54:28,438 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36199] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4eb5d37a; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:28,438 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f91831a12657ef5da469c59d1a700126/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,438 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36199] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 23:54:28,438 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/9aaa63f230a54ddc8a10bd9042049011] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/.tmp, totalSize=6.8 K 2023-02-08 23:54:28,438 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36199] regionserver.HStore(1912): f91831a12657ef5da469c59d1a700126/cf2 is initiating major compaction (all files) 2023-02-08 23:54:28,438 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36199] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2923ed2f; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:28,438 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f91831a12657ef5da469c59d1a700126/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,438 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/9bf35bc1c4e14a0ba16ae79db1893fe8] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/.tmp, totalSize=6.8 K 2023-02-08 23:54:28,438 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 9aaa63f230a54ddc8a10bd9042049011, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675900465767 2023-02-08 23:54:28,438 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting 9bf35bc1c4e14a0ba16ae79db1893fe8, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675900465767 2023-02-08 23:54:28,440 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36199] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,440 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36199] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 23:54:28,440 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36199] regionserver.HStore(1912): 1f5b68527c02157d9f50f31363d383eb/cf1 is initiating major compaction (all files) 2023-02-08 23:54:28,440 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36199] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@47ded67d; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:54:28,440 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36199] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 23:54:28,440 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36199] regionserver.HStore(1912): 1f5b68527c02157d9f50f31363d383eb/cf2 is initiating major compaction (all files) 2023-02-08 23:54:28,440 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36199] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3b09c44; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-08 23:54:28,441 INFO [Listener at localhost/37973] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-08 23:54:28,442 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] throttle.PressureAwareThroughputController(145): f91831a12657ef5da469c59d1a700126#cf1#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:28,443 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] throttle.PressureAwareThroughputController(145): f91831a12657ef5da469c59d1a700126#cf2#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:28,460 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36199] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. is not mergeable because it has references 2023-02-08 23:54:28,461 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36199] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. is not mergeable because it has references 2023-02-08 23:54:28,465 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:54:28,466 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/.tmp/cf2/6146949636064746a7a923e34d66a94c as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/6146949636064746a7a923e34d66a94c 2023-02-08 23:54:28,466 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/.tmp/cf1/fdd4cd815d274ea2af7099abda80f605 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/fdd4cd815d274ea2af7099abda80f605 2023-02-08 23:54:28,470 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in f91831a12657ef5da469c59d1a700126/cf2 of f91831a12657ef5da469c59d1a700126 into 6146949636064746a7a923e34d66a94c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:28,470 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f91831a12657ef5da469c59d1a700126: 2023-02-08 23:54:28,471 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in f91831a12657ef5da469c59d1a700126/cf1 of f91831a12657ef5da469c59d1a700126 into fdd4cd815d274ea2af7099abda80f605(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:28,471 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f91831a12657ef5da469c59d1a700126: 2023-02-08 23:54:28,472 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675900468438; duration=0sec 2023-02-08 23:54:28,472 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675900468437; duration=0sec 2023-02-08 23:54:28,473 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-08 23:54:28,473 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-08 23:54:28,473 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1f5b68527c02157d9f50f31363d383eb/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,473 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1f5b68527c02157d9f50f31363d383eb/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,473 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/d9db015f2bc64bed9b2480c34e3ee9c2] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/.tmp, totalSize=5.0 K 2023-02-08 23:54:28,473 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/753505c5d4d24c47b38f69b83a230c0d] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/.tmp, totalSize=5.0 K 2023-02-08 23:54:28,473 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] compactions.Compactor(207): Compacting d9db015f2bc64bed9b2480c34e3ee9c2, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-08 23:54:28,473 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] compactions.Compactor(207): Compacting 753505c5d4d24c47b38f69b83a230c0d, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-08 23:54:28,480 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] throttle.PressureAwareThroughputController(145): 1f5b68527c02157d9f50f31363d383eb#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:28,480 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1f5b68527c02157d9f50f31363d383eb#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:28,506 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 5158df84bd474ee1b2f06d42634ccb08: NONE, but ROW specified in column family configuration 2023-02-08 23:54:28,506 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 61393b10a10a404086231b274d00fcc0: NONE, but ROW specified in column family configuration 2023-02-08 23:54:28,507 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/.tmp/cf2/5158df84bd474ee1b2f06d42634ccb08 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/5158df84bd474ee1b2f06d42634ccb08 2023-02-08 23:54:28,507 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/.tmp/cf1/61393b10a10a404086231b274d00fcc0 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/61393b10a10a404086231b274d00fcc0 2023-02-08 23:54:28,511 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 5158df84bd474ee1b2f06d42634ccb08: NONE, but ROW specified in column family configuration 2023-02-08 23:54:28,512 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 61393b10a10a404086231b274d00fcc0: NONE, but ROW specified in column family configuration 2023-02-08 23:54:28,513 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 1f5b68527c02157d9f50f31363d383eb/cf2 of 1f5b68527c02157d9f50f31363d383eb into 5158df84bd474ee1b2f06d42634ccb08(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:28,513 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1f5b68527c02157d9f50f31363d383eb: 2023-02-08 23:54:28,513 INFO [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675900468440; duration=0sec 2023-02-08 23:54:28,513 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 1f5b68527c02157d9f50f31363d383eb/cf1 of 1f5b68527c02157d9f50f31363d383eb into 61393b10a10a404086231b274d00fcc0(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:28,513 DEBUG [RS:1;jenkins-hbase4:36199-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:28,513 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1f5b68527c02157d9f50f31363d383eb: 2023-02-08 23:54:28,513 INFO [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675900468440; duration=0sec 2023-02-08 23:54:28,513 DEBUG [RS:1;jenkins-hbase4:36199-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:28,570 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36199] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. is not mergeable because it has references 2023-02-08 23:54:28,571 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36199] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. is not mergeable because it has references 2023-02-08 23:54:28,579 DEBUG [Listener at localhost/37973] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-08 23:54:28,581 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:28,581 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:28,581 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:28,582 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:28,582 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:28,583 INFO [Listener at localhost/37973] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:28,583 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:28,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:28,585 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:28,586 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:28,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 23:54:28,586 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900468586"}]},"ts":"1675900468586"} 2023-02-08 23:54:28,587 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-08 23:54:28,589 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-08 23:54:28,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=f91831a12657ef5da469c59d1a700126, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f5b68527c02157d9f50f31363d383eb, UNASSIGN}] 2023-02-08 23:54:28,591 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f5b68527c02157d9f50f31363d383eb, UNASSIGN 2023-02-08 23:54:28,591 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=1f5b68527c02157d9f50f31363d383eb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:28,591 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675900468591"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900468591"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900468591"}]},"ts":"1675900468591"} 2023-02-08 23:54:28,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 1f5b68527c02157d9f50f31363d383eb, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:28,594 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=f91831a12657ef5da469c59d1a700126, UNASSIGN 2023-02-08 23:54:28,595 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=f91831a12657ef5da469c59d1a700126, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:28,595 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675900468595"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900468595"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900468595"}]},"ts":"1675900468595"} 2023-02-08 23:54:28,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure f91831a12657ef5da469c59d1a700126, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:28,686 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:28,687 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:28,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 23:54:28,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f91831a12657ef5da469c59d1a700126 2023-02-08 23:54:28,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f91831a12657ef5da469c59d1a700126, disabling compactions & flushes 2023-02-08 23:54:28,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. after waiting 0 ms 2023-02-08 23:54:28,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,745 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/07e210660ce448e39e9b9091cc8c6a39.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/07e210660ce448e39e9b9091cc8c6a39-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/662156a8d4454e3f9b39bd5706bea215.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/662156a8d4454e3f9b39bd5706bea215-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/7f17d7b5808247ecbb447fefd04dd77d.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/7f17d7b5808247ecbb447fefd04dd77d-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-9f37c71614b644028c60a208ec2dae92, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-47c0e414716643c0915dd58669faf777, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-b1cd1edaaa404cc6bc0ec1a597a04c22, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/9aaa63f230a54ddc8a10bd9042049011, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/a167a9ec1531417b957c21005dc49753-bottom] to archive 2023-02-08 23:54:28,746 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:28,747 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/07e210660ce448e39e9b9091cc8c6a39.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/07e210660ce448e39e9b9091cc8c6a39.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,749 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/662156a8d4454e3f9b39bd5706bea215.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/662156a8d4454e3f9b39bd5706bea215.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,750 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/7f17d7b5808247ecbb447fefd04dd77d.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/7f17d7b5808247ecbb447fefd04dd77d.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,750 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-9f37c71614b644028c60a208ec2dae92 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-9f37c71614b644028c60a208ec2dae92 2023-02-08 23:54:28,751 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-47c0e414716643c0915dd58669faf777 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-47c0e414716643c0915dd58669faf777 2023-02-08 23:54:28,752 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-b1cd1edaaa404cc6bc0ec1a597a04c22 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-b1cd1edaaa404cc6bc0ec1a597a04c22 2023-02-08 23:54:28,753 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/9aaa63f230a54ddc8a10bd9042049011 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/9aaa63f230a54ddc8a10bd9042049011 2023-02-08 23:54:28,754 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,756 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/3d18efc2303f4e3ea2ba3fc349726866.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3d18efc2303f4e3ea2ba3fc349726866-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/f5c24e92b1274a82a4db8a9b84b51be9.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f5c24e92b1274a82a4db8a9b84b51be9-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/be1ff474bd5c426faa85b294d8836664.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/be1ff474bd5c426faa85b294d8836664-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-e9ed93d9f2cb4510856a03cfa143dbba, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d42ed95ed42045a3a02129eb8f151fa2, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ad0267ee4b044ef1a66b9bcf4a157841, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/9bf35bc1c4e14a0ba16ae79db1893fe8, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90-bottom] to archive 2023-02-08 23:54:28,756 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:28,757 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/3d18efc2303f4e3ea2ba3fc349726866.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/3d18efc2303f4e3ea2ba3fc349726866.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,758 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/f5c24e92b1274a82a4db8a9b84b51be9.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/f5c24e92b1274a82a4db8a9b84b51be9.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,759 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/be1ff474bd5c426faa85b294d8836664.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/be1ff474bd5c426faa85b294d8836664.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,760 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-e9ed93d9f2cb4510856a03cfa143dbba to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-e9ed93d9f2cb4510856a03cfa143dbba 2023-02-08 23:54:28,761 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d42ed95ed42045a3a02129eb8f151fa2 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d42ed95ed42045a3a02129eb8f151fa2 2023-02-08 23:54:28,762 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ad0267ee4b044ef1a66b9bcf4a157841 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ad0267ee4b044ef1a66b9bcf4a157841 2023-02-08 23:54:28,763 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/9bf35bc1c4e14a0ba16ae79db1893fe8 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/9bf35bc1c4e14a0ba16ae79db1893fe8 2023-02-08 23:54:28,764 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-08 23:54:28,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:28,769 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126. 2023-02-08 23:54:28,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f91831a12657ef5da469c59d1a700126: Waiting for close lock at 1675900468744Running coprocessor pre-close hooks at 1675900468745 (+1 ms)Disabling compacts and flushes for region at 1675900468745Disabling writes for close at 1675900468745Writing region close event to WAL at 1675900468765 (+20 ms)Running coprocessor post-close hooks at 1675900468768 (+3 ms)Closed at 1675900468769 (+1 ms) 2023-02-08 23:54:28,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f91831a12657ef5da469c59d1a700126 2023-02-08 23:54:28,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:28,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1f5b68527c02157d9f50f31363d383eb, disabling compactions & flushes 2023-02-08 23:54:28,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. after waiting 0 ms 2023-02-08 23:54:28,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,771 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=f91831a12657ef5da469c59d1a700126, regionState=CLOSED 2023-02-08 23:54:28,771 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675900468771"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900468771"}]},"ts":"1675900468771"} 2023-02-08 23:54:28,772 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/07e210660ce448e39e9b9091cc8c6a39.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/07e210660ce448e39e9b9091cc8c6a39-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/662156a8d4454e3f9b39bd5706bea215.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/662156a8d4454e3f9b39bd5706bea215-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/7f17d7b5808247ecbb447fefd04dd77d.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/7f17d7b5808247ecbb447fefd04dd77d-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/a167a9ec1531417b957c21005dc49753-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-1d802083a3c24e729fd7cc4d28b65e67, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ebe3c354544a455ea14ab74024a1a9b1, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/d9db015f2bc64bed9b2480c34e3ee9c2, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-25827684de8d44188b27dfe977104e72] to archive 2023-02-08 23:54:28,772 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:28,774 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-08 23:54:28,774 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure f91831a12657ef5da469c59d1a700126, server=jenkins-hbase4.apache.org,36199,1675900441120 in 176 msec 2023-02-08 23:54:28,774 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/07e210660ce448e39e9b9091cc8c6a39.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/07e210660ce448e39e9b9091cc8c6a39.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,775 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=f91831a12657ef5da469c59d1a700126, UNASSIGN in 184 msec 2023-02-08 23:54:28,775 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/662156a8d4454e3f9b39bd5706bea215.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/662156a8d4454e3f9b39bd5706bea215.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,776 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/7f17d7b5808247ecbb447fefd04dd77d.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/7f17d7b5808247ecbb447fefd04dd77d.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,777 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/a167a9ec1531417b957c21005dc49753.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,778 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-1d802083a3c24e729fd7cc4d28b65e67 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-1d802083a3c24e729fd7cc4d28b65e67 2023-02-08 23:54:28,779 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ebe3c354544a455ea14ab74024a1a9b1 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-ebe3c354544a455ea14ab74024a1a9b1 2023-02-08 23:54:28,780 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/d9db015f2bc64bed9b2480c34e3ee9c2 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/d9db015f2bc64bed9b2480c34e3ee9c2 2023-02-08 23:54:28,781 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-25827684de8d44188b27dfe977104e72 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-25827684de8d44188b27dfe977104e72 2023-02-08 23:54:28,782 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/3d18efc2303f4e3ea2ba3fc349726866.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3d18efc2303f4e3ea2ba3fc349726866-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/f5c24e92b1274a82a4db8a9b84b51be9.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f5c24e92b1274a82a4db8a9b84b51be9-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/be1ff474bd5c426faa85b294d8836664.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/be1ff474bd5c426faa85b294d8836664-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d5d5d0aa7c004461a1b53564f6b76bea, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-3fe42b5e5f5f452abeeb859d3a43b350, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/753505c5d4d24c47b38f69b83a230c0d, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-f538e988a65f4cfe95e8c5898426419d] to archive 2023-02-08 23:54:28,783 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:28,784 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/3d18efc2303f4e3ea2ba3fc349726866.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/3d18efc2303f4e3ea2ba3fc349726866.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,785 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/f5c24e92b1274a82a4db8a9b84b51be9.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/f5c24e92b1274a82a4db8a9b84b51be9.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,786 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/be1ff474bd5c426faa85b294d8836664.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/be1ff474bd5c426faa85b294d8836664.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,788 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90.0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,789 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d5d5d0aa7c004461a1b53564f6b76bea to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-d5d5d0aa7c004461a1b53564f6b76bea 2023-02-08 23:54:28,790 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-3fe42b5e5f5f452abeeb859d3a43b350 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-3fe42b5e5f5f452abeeb859d3a43b350 2023-02-08 23:54:28,791 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/753505c5d4d24c47b38f69b83a230c0d to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/753505c5d4d24c47b38f69b83a230c0d 2023-02-08 23:54:28,792 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-f538e988a65f4cfe95e8c5898426419d to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/testSplitTableRegionDeletedRowsDaughter=0a59441954132eb44621ade02029d4dc-f538e988a65f4cfe95e8c5898426419d 2023-02-08 23:54:28,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-08 23:54:28,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:28,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb. 2023-02-08 23:54:28,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1f5b68527c02157d9f50f31363d383eb: Waiting for close lock at 1675900468771Running coprocessor pre-close hooks at 1675900468771Disabling compacts and flushes for region at 1675900468771Disabling writes for close at 1675900468771Writing region close event to WAL at 1675900468793 (+22 ms)Running coprocessor post-close hooks at 1675900468803 (+10 ms)Closed at 1675900468804 (+1 ms) 2023-02-08 23:54:28,806 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:28,806 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=1f5b68527c02157d9f50f31363d383eb, regionState=CLOSED 2023-02-08 23:54:28,806 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675900468806"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900468806"}]},"ts":"1675900468806"} 2023-02-08 23:54:28,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-08 23:54:28,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 1f5b68527c02157d9f50f31363d383eb, server=jenkins-hbase4.apache.org,36199,1675900441120 in 216 msec 2023-02-08 23:54:28,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-08 23:54:28,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f5b68527c02157d9f50f31363d383eb, UNASSIGN in 219 msec 2023-02-08 23:54:28,812 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900468812"}]},"ts":"1675900468812"} 2023-02-08 23:54:28,813 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-08 23:54:28,815 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-08 23:54:28,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 232 msec 2023-02-08 23:54:28,888 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:28,888 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:28,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 23:54:28,889 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-08 23:54:28,889 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:28,889 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:28,890 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:28,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:28,892 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:28,893 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:28,893 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:28,893 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:28,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-08 23:54:28,897 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,897 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:28,897 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126 2023-02-08 23:54:28,899 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/recovered.edits] 2023-02-08 23:54:28,899 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/recovered.edits] 2023-02-08 23:54:28,899 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/recovered.edits] 2023-02-08 23:54:28,902 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/fdd4cd815d274ea2af7099abda80f605 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf1/fdd4cd815d274ea2af7099abda80f605 2023-02-08 23:54:28,902 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/61393b10a10a404086231b274d00fcc0 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf1/61393b10a10a404086231b274d00fcc0 2023-02-08 23:54:28,903 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/07e210660ce448e39e9b9091cc8c6a39 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/07e210660ce448e39e9b9091cc8c6a39 2023-02-08 23:54:28,904 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/1d802083a3c24e729fd7cc4d28b65e67 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/1d802083a3c24e729fd7cc4d28b65e67 2023-02-08 23:54:28,905 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/5158df84bd474ee1b2f06d42634ccb08 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/cf2/5158df84bd474ee1b2f06d42634ccb08 2023-02-08 23:54:28,905 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/6146949636064746a7a923e34d66a94c to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/cf2/6146949636064746a7a923e34d66a94c 2023-02-08 23:54:28,905 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/25827684de8d44188b27dfe977104e72 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/25827684de8d44188b27dfe977104e72 2023-02-08 23:54:28,907 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/47c0e414716643c0915dd58669faf777 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/47c0e414716643c0915dd58669faf777 2023-02-08 23:54:28,908 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/recovered.edits/113.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb/recovered.edits/113.seqid 2023-02-08 23:54:28,908 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/662156a8d4454e3f9b39bd5706bea215 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/662156a8d4454e3f9b39bd5706bea215 2023-02-08 23:54:28,908 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/recovered.edits/113.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126/recovered.edits/113.seqid 2023-02-08 23:54:28,909 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:54:28,909 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/f91831a12657ef5da469c59d1a700126 2023-02-08 23:54:28,909 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/7f17d7b5808247ecbb447fefd04dd77d to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/7f17d7b5808247ecbb447fefd04dd77d 2023-02-08 23:54:28,911 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/9f37c71614b644028c60a208ec2dae92 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/9f37c71614b644028c60a208ec2dae92 2023-02-08 23:54:28,912 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/a167a9ec1531417b957c21005dc49753 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/a167a9ec1531417b957c21005dc49753 2023-02-08 23:54:28,913 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/b1cd1edaaa404cc6bc0ec1a597a04c22 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/b1cd1edaaa404cc6bc0ec1a597a04c22 2023-02-08 23:54:28,914 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/ebe3c354544a455ea14ab74024a1a9b1 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf1/ebe3c354544a455ea14ab74024a1a9b1 2023-02-08 23:54:28,916 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3d18efc2303f4e3ea2ba3fc349726866 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3d18efc2303f4e3ea2ba3fc349726866 2023-02-08 23:54:28,917 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3fe42b5e5f5f452abeeb859d3a43b350 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/3fe42b5e5f5f452abeeb859d3a43b350 2023-02-08 23:54:28,919 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/4e9f007c94ea4bcc998e5ee8e1e51e90 2023-02-08 23:54:28,920 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/ad0267ee4b044ef1a66b9bcf4a157841 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/ad0267ee4b044ef1a66b9bcf4a157841 2023-02-08 23:54:28,921 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/be1ff474bd5c426faa85b294d8836664 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/be1ff474bd5c426faa85b294d8836664 2023-02-08 23:54:28,922 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d42ed95ed42045a3a02129eb8f151fa2 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d42ed95ed42045a3a02129eb8f151fa2 2023-02-08 23:54:28,923 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d5d5d0aa7c004461a1b53564f6b76bea to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/d5d5d0aa7c004461a1b53564f6b76bea 2023-02-08 23:54:28,923 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/e9ed93d9f2cb4510856a03cfa143dbba to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/e9ed93d9f2cb4510856a03cfa143dbba 2023-02-08 23:54:28,924 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f538e988a65f4cfe95e8c5898426419d to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f538e988a65f4cfe95e8c5898426419d 2023-02-08 23:54:28,925 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f5c24e92b1274a82a4db8a9b84b51be9 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/cf2/f5c24e92b1274a82a4db8a9b84b51be9 2023-02-08 23:54:28,927 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/recovered.edits/106.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc/recovered.edits/106.seqid 2023-02-08 23:54:28,928 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0a59441954132eb44621ade02029d4dc 2023-02-08 23:54:28,928 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-08 23:54:28,930 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:28,932 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-08 23:54:28,933 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-08 23:54:28,934 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:28,934 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-08 23:54:28,934 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900468934"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:28,934 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900468934"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:28,934 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900468934"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:28,936 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 23:54:28,936 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0a59441954132eb44621ade02029d4dc, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675900465262.0a59441954132eb44621ade02029d4dc.', STARTKEY => '', ENDKEY => ''}, {ENCODED => f91831a12657ef5da469c59d1a700126, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675900467420.f91831a12657ef5da469c59d1a700126.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 1f5b68527c02157d9f50f31363d383eb, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675900467420.1f5b68527c02157d9f50f31363d383eb.', STARTKEY => '60', ENDKEY => ''}] 2023-02-08 23:54:28,936 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-08 23:54:28,936 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675900468936"}]},"ts":"9223372036854775807"} 2023-02-08 23:54:28,937 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-08 23:54:28,939 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:54:28,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 49 msec 2023-02-08 23:54:28,994 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:28,994 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:28,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-08 23:54:28,995 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-08 23:54:29,009 INFO [Listener at localhost/37973] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=405 (was 401) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:51426 [Waiting for operation #35] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:39338 [Waiting for operation #53] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:41816 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:41716 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:58170 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:58436 [Waiting for operation #31] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:52572 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:36626 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:58898 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:58448 [Waiting for operation #45] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:41776 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:36612 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:42938 [Waiting for operation #31] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:52846 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:52826 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-750809158_15 at /127.0.0.1:51318 [Waiting for operation #34] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=849 (was 849), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=254 (was 268), ProcessCount=168 (was 168), AvailableMemoryMB=3296 (was 3354) 2023-02-08 23:54:29,024 INFO [Listener at localhost/37973] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=405, OpenFileDescriptor=849, MaxFileDescriptor=60000, SystemLoadAverage=254, ProcessCount=168, AvailableMemoryMB=3296 2023-02-08 23:54:29,024 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:54:29,024 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:29,024 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:29,028 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 23:54:29,028 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 23:54:29,029 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:54:29,031 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:54:29,031 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:54:29,033 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:29,033 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322 empty. 2023-02-08 23:54:29,033 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:29,033 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-08 23:54:29,044 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:29,045 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4a9ac7b252881a6ce15fd79f0bb87322, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp 2023-02-08 23:54:29,052 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:29,052 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 4a9ac7b252881a6ce15fd79f0bb87322, disabling compactions & flushes 2023-02-08 23:54:29,052 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:29,052 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:29,053 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. after waiting 0 ms 2023-02-08 23:54:29,053 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:29,053 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:29,053 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 4a9ac7b252881a6ce15fd79f0bb87322: Waiting for close lock at 1675900469052Disabling compacts and flushes for region at 1675900469052Disabling writes for close at 1675900469053 (+1 ms)Writing region close event to WAL at 1675900469053Closed at 1675900469053 2023-02-08 23:54:29,055 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:54:29,055 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900469055"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900469055"}]},"ts":"1675900469055"} 2023-02-08 23:54:29,057 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:54:29,057 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:54:29,058 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900469058"}]},"ts":"1675900469058"} 2023-02-08 23:54:29,059 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-08 23:54:29,061 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:54:29,062 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:54:29,062 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:54:29,062 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:54:29,062 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 23:54:29,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=4a9ac7b252881a6ce15fd79f0bb87322, ASSIGN}] 2023-02-08 23:54:29,064 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=4a9ac7b252881a6ce15fd79f0bb87322, ASSIGN 2023-02-08 23:54:29,064 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=4a9ac7b252881a6ce15fd79f0bb87322, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=false 2023-02-08 23:54:29,215 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:54:29,217 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=4a9ac7b252881a6ce15fd79f0bb87322, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:29,218 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900469217"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900469217"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900469217"}]},"ts":"1675900469217"} 2023-02-08 23:54:29,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 4a9ac7b252881a6ce15fd79f0bb87322, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:29,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:29,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4a9ac7b252881a6ce15fd79f0bb87322, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:29,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:29,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:29,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:29,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:29,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:29,375 INFO [StoreOpener-4a9ac7b252881a6ce15fd79f0bb87322-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:29,377 DEBUG [StoreOpener-4a9ac7b252881a6ce15fd79f0bb87322-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/cf1 2023-02-08 23:54:29,377 DEBUG [StoreOpener-4a9ac7b252881a6ce15fd79f0bb87322-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/cf1 2023-02-08 23:54:29,377 INFO [StoreOpener-4a9ac7b252881a6ce15fd79f0bb87322-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4a9ac7b252881a6ce15fd79f0bb87322 columnFamilyName cf1 2023-02-08 23:54:29,378 INFO [StoreOpener-4a9ac7b252881a6ce15fd79f0bb87322-1] regionserver.HStore(310): Store=4a9ac7b252881a6ce15fd79f0bb87322/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:29,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:29,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:29,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:29,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:29,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4a9ac7b252881a6ce15fd79f0bb87322; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60784510, jitterRate=-0.0942402184009552}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:54:29,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4a9ac7b252881a6ce15fd79f0bb87322: Running coprocessor pre-open hook at 1675900469374Writing region info on filesystem at 1675900469374Initializing all the Stores at 1675900469375 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900469375Cleaning up temporary data from old regions at 1675900469379 (+4 ms)Cleaning up detritus from prior splits at 1675900469380 (+1 ms)Running coprocessor post-open hooks at 1675900469384 (+4 ms)Region opened successfully at 1675900469384 2023-02-08 23:54:29,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322., pid=87, masterSystemTime=1675900469371 2023-02-08 23:54:29,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:29,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:29,387 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=4a9ac7b252881a6ce15fd79f0bb87322, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:29,387 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900469387"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900469387"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900469387"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900469387"}]},"ts":"1675900469387"} 2023-02-08 23:54:29,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-08 23:54:29,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 4a9ac7b252881a6ce15fd79f0bb87322, server=jenkins-hbase4.apache.org,36199,1675900441120 in 170 msec 2023-02-08 23:54:29,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-08 23:54:29,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=4a9ac7b252881a6ce15fd79f0bb87322, ASSIGN in 328 msec 2023-02-08 23:54:29,392 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:54:29,392 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900469392"}]},"ts":"1675900469392"} 2023-02-08 23:54:29,393 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-08 23:54:29,397 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:54:29,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 368 msec 2023-02-08 23:54:29,530 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:29,530 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:29,532 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:29,532 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:29,539 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-08 23:54:29,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:54:29,558 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-08 23:54:29,559 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:29,559 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-08 23:54:29,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-08 23:54:29,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=4a9ac7b252881a6ce15fd79f0bb87322, REOPEN/MOVE}] 2023-02-08 23:54:29,575 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=4a9ac7b252881a6ce15fd79f0bb87322, REOPEN/MOVE 2023-02-08 23:54:29,575 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=4a9ac7b252881a6ce15fd79f0bb87322, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:29,575 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900469575"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900469575"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900469575"}]},"ts":"1675900469575"} 2023-02-08 23:54:29,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 4a9ac7b252881a6ce15fd79f0bb87322, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:29,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:29,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4a9ac7b252881a6ce15fd79f0bb87322, disabling compactions & flushes 2023-02-08 23:54:29,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:29,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:29,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. after waiting 0 ms 2023-02-08 23:54:29,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:29,733 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-08 23:54:29,733 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:54:29,734 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:29,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4a9ac7b252881a6ce15fd79f0bb87322: Waiting for close lock at 1675900469728Running coprocessor pre-close hooks at 1675900469729 (+1 ms)Disabling compacts and flushes for region at 1675900469729Disabling writes for close at 1675900469730 (+1 ms)Writing region close event to WAL at 1675900469730Running coprocessor post-close hooks at 1675900469733 (+3 ms)Closed at 1675900469734 (+1 ms) 2023-02-08 23:54:29,734 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 4a9ac7b252881a6ce15fd79f0bb87322 to self. 2023-02-08 23:54:29,735 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:29,736 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=4a9ac7b252881a6ce15fd79f0bb87322, regionState=CLOSED 2023-02-08 23:54:29,736 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900469736"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900469736"}]},"ts":"1675900469736"} 2023-02-08 23:54:29,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-08 23:54:29,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 4a9ac7b252881a6ce15fd79f0bb87322, server=jenkins-hbase4.apache.org,36199,1675900441120 in 160 msec 2023-02-08 23:54:29,739 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=4a9ac7b252881a6ce15fd79f0bb87322, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36199,1675900441120; forceNewPlan=false, retain=true 2023-02-08 23:54:29,890 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:54:29,890 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=4a9ac7b252881a6ce15fd79f0bb87322, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:29,890 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900469890"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900469890"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900469890"}]},"ts":"1675900469890"} 2023-02-08 23:54:29,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 4a9ac7b252881a6ce15fd79f0bb87322, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:54:30,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:30,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4a9ac7b252881a6ce15fd79f0bb87322, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:30,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:30,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:30,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:30,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:30,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:30,048 INFO [StoreOpener-4a9ac7b252881a6ce15fd79f0bb87322-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:30,049 DEBUG [StoreOpener-4a9ac7b252881a6ce15fd79f0bb87322-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/cf1 2023-02-08 23:54:30,049 DEBUG [StoreOpener-4a9ac7b252881a6ce15fd79f0bb87322-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/cf1 2023-02-08 23:54:30,050 INFO [StoreOpener-4a9ac7b252881a6ce15fd79f0bb87322-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4a9ac7b252881a6ce15fd79f0bb87322 columnFamilyName cf1 2023-02-08 23:54:30,050 INFO [StoreOpener-4a9ac7b252881a6ce15fd79f0bb87322-1] regionserver.HStore(310): Store=4a9ac7b252881a6ce15fd79f0bb87322/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:30,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:30,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:30,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:54:30,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4a9ac7b252881a6ce15fd79f0bb87322; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65563495, jitterRate=-0.02302779257297516}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:54:30,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4a9ac7b252881a6ce15fd79f0bb87322: Running coprocessor pre-open hook at 1675900470047Writing region info on filesystem at 1675900470047Initializing all the Stores at 1675900470048 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900470048Cleaning up temporary data from old regions at 1675900470053 (+5 ms)Cleaning up detritus from prior splits at 1675900470054 (+1 ms)Running coprocessor post-open hooks at 1675900470057 (+3 ms)Region opened successfully at 1675900470057 2023-02-08 23:54:30,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322., pid=92, masterSystemTime=1675900470043 2023-02-08 23:54:30,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:30,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:54:30,060 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=4a9ac7b252881a6ce15fd79f0bb87322, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:54:30,060 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900470060"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900470060"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900470060"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900470060"}]},"ts":"1675900470060"} 2023-02-08 23:54:30,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-08 23:54:30,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 4a9ac7b252881a6ce15fd79f0bb87322, server=jenkins-hbase4.apache.org,36199,1675900441120 in 169 msec 2023-02-08 23:54:30,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-08 23:54:30,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=4a9ac7b252881a6ce15fd79f0bb87322, REOPEN/MOVE in 490 msec 2023-02-08 23:54:30,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-08 23:54:30,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 507 msec 2023-02-08 23:54:30,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1, ASSIGN}] 2023-02-08 23:54:30,074 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1, ASSIGN 2023-02-08 23:54:30,075 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-08 23:54:30,225 DEBUG [jenkins-hbase4:37353] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:54:30,226 DEBUG [jenkins-hbase4:37353] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:54:30,227 DEBUG [jenkins-hbase4:37353] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:54:30,227 DEBUG [jenkins-hbase4:37353] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:54:30,227 DEBUG [jenkins-hbase4:37353] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-08 23:54:30,228 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=7271d73f1b2300d37c9209b7cecf4ba1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:30,229 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900470228"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1675900470228"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675900470228"}]},"ts":"1675900470228"} 2023-02-08 23:54:30,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 7271d73f1b2300d37c9209b7cecf4ba1, server=jenkins-hbase4.apache.org,36713,1675900441074}] 2023-02-08 23:54:30,384 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:54:30,385 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 23:54:30,387 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46642, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 23:54:30,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1. 2023-02-08 23:54:30,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7271d73f1b2300d37c9209b7cecf4ba1, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-08 23:54:30,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:54:30,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 7271d73f1b2300d37c9209b7cecf4ba1 2023-02-08 23:54:30,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:30,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7271d73f1b2300d37c9209b7cecf4ba1 2023-02-08 23:54:30,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7271d73f1b2300d37c9209b7cecf4ba1 2023-02-08 23:54:30,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:30,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:30,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:30,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:30,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:30,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:30,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:31,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:31,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:31,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:31,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:31,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:31,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:31,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:31,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:31,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:31,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:32,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:32,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:32,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:32,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:32,489 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-08 23:54:32,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:32,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:32,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:32,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:32,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:32,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:33,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:33,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:33,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:33,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:33,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:33,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:33,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:33,702 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:54:33,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:33,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:34,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:34,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:34,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:34,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:34,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:34,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:34,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:34,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:34,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:34,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:35,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:35,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:35,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:35,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:35,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:35,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:35,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:35,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:35,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:35,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:36,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:36,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:36,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:36,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:36,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:36,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:36,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:36,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:36,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:36,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:37,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:37,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:37,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:37,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:37,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:37,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:37,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:37,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:37,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:37,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:38,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:38,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:38,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:38,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:38,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:38,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:38,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:38,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:38,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:38,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:39,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:39,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:39,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:39,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:39,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:39,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:39,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:39,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:39,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:39,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:40,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:40,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:40,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:40,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:40,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:40,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:40,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:40,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:40,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:40,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:41,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:41,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:41,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:41,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:41,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:41,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:41,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:41,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:41,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:41,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:42,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:42,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:42,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:42,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:42,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:42,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:42,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:42,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:42,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:42,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:43,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:43,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:43,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:43,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:43,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:43,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:43,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:43,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:43,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:43,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:44,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:44,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:44,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:44,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:44,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:44,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:44,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:44,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:44,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:44,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:45,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:45,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:45,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:45,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:45,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:45,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:45,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:45,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:45,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:45,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:46,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:46,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:46,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:46,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:46,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:46,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:46,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:46,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:46,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:46,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:47,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:47,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:47,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:47,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:47,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:47,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:47,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:47,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:47,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:47,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:48,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:48,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:48,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:48,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:48,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:48,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:48,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:48,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:48,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:48,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:49,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:49,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:49,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:49,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:49,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:49,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:49,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:49,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:49,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:49,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:50,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:50,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:50,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:50,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:50,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:50,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:50,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:50,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:50,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:50,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:51,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:51,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:51,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:51,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:51,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:51,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:51,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:51,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:51,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:51,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:52,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:52,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:52,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:52,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:52,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:52,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:52,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:52,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:52,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:52,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:53,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:53,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:53,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:53,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:53,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:53,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:53,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:53,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:53,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:53,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:54,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:54,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:54,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:54,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:54,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:54,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:54,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:54,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:54,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:54,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:55,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:55,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:55,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:55,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:55,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:55,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:55,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:55,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:55,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:55,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:56,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:56,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:56,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:56,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:56,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:56,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:56,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:56,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:56,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:56,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:57,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:57,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:57,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:57,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:57,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:57,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:57,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:57,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:57,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:57,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:58,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:58,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:58,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:58,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:58,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:58,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:58,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:58,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:58,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:58,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:59,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:59,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:59,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:59,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:59,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:59,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:59,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:59,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:59,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:54:59,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:00,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:00,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:00,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:00,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:00,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:00,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:00,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:00,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:00,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:00,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:01,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:01,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:01,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:01,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:01,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:01,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:01,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:01,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:01,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:01,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:02,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:02,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:02,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:02,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:02,398 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 23:55:02,438 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 23:55:02,442 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 23:55:02,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:02,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:02,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:02,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:02,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:02,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:03,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:03,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:03,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:03,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:03,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:03,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:03,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:03,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:03,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:03,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:04,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:04,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:04,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:04,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:04,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:04,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:04,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:04,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:04,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:04,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:05,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:05,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:05,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:05,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:05,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:05,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:05,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:05,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:05,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:05,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:06,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:06,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:06,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:06,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:06,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:06,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:06,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:06,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:06,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:06,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:07,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:07,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:07,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:07,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:07,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:07,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:07,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:07,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:07,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:07,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:08,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:08,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:08,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:08,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:08,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:08,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:08,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:08,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:08,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:08,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:09,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:09,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:09,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:09,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:09,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:09,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:09,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:09,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:09,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:09,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:10,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:10,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:10,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:10,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:10,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:10,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:10,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:10,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:10,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:10,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:11,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:11,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:11,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:11,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:11,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:11,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:11,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:11,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:11,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:11,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:12,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:12,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:12,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:12,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:12,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:12,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:12,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:12,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:12,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:12,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:13,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:13,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:13,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:13,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:13,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:13,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:13,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:13,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:13,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:13,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:14,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:14,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:14,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:14,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:14,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:14,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:14,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:14,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:14,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:14,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:15,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:15,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:15,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:15,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:15,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:15,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:15,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:15,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:15,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:15,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:16,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:16,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:16,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:16,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:16,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:16,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:16,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:16,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:16,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:16,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:17,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:17,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:17,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:17,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:17,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:17,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:17,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:17,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:17,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:17,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:18,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:18,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:18,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:18,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:18,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:18,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:18,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:18,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:18,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:18,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:19,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:19,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:19,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:19,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:19,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:19,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:19,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:19,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:19,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:19,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:20,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:20,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:20,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:20,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:20,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:20,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:20,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:20,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:20,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:20,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:21,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:21,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:21,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:21,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:21,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:21,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:21,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:21,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:21,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:21,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:22,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:22,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:22,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:22,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:22,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:22,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:22,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:22,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:22,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:22,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:23,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:23,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:23,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:23,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:23,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:23,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:23,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:23,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:23,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:24,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:24,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:24,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:24,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:24,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:24,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:24,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:24,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:24,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:24,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:25,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:25,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:25,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:25,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:25,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:25,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:25,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:25,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:25,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:25,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:26,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:26,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:26,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:26,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:26,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:26,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:26,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:26,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:26,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:26,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:27,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:27,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:27,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:27,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:27,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:27,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:27,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:27,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:27,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:27,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:28,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:28,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:28,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:28,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:28,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:28,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:28,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:28,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:28,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:28,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:29,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:29,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:29,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:29,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:29,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:29,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:29,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:29,650 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:29,650 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:29,654 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:29,654 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:29,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 23:55:29,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:29,755 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:29,755 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:29,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 23:55:29,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:29,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:29,956 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:29,956 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:29,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 23:55:30,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:30,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:30,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:30,257 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:30,257 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:30,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 23:55:30,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:30,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:30,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:30,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:30,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:30,758 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:30,759 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:30,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 23:55:30,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:30,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:31,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:31,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:31,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:31,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:31,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:31,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:31,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:31,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:31,761 DEBUG [Listener at localhost/37973] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:31,762 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:31,763 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:31,763 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:31,763 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:31,765 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 4a9ac7b252881a6ce15fd79f0bb87322, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:55:31,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 exec-time=6 msec 2023-02-08 23:55:31,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:31,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 23:55:32,017 INFO [StoreOpener-7271d73f1b2300d37c9209b7cecf4ba1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7271d73f1b2300d37c9209b7cecf4ba1 2023-02-08 23:55:32,021 DEBUG [StoreOpener-7271d73f1b2300d37c9209b7cecf4ba1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/cf1 2023-02-08 23:55:32,022 DEBUG [StoreOpener-7271d73f1b2300d37c9209b7cecf4ba1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/cf1 2023-02-08 23:55:32,022 INFO [StoreOpener-7271d73f1b2300d37c9209b7cecf4ba1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7271d73f1b2300d37c9209b7cecf4ba1 columnFamilyName cf1 2023-02-08 23:55:32,023 INFO [StoreOpener-7271d73f1b2300d37c9209b7cecf4ba1-1] regionserver.HStore(310): Store=7271d73f1b2300d37c9209b7cecf4ba1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:32,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7271d73f1b2300d37c9209b7cecf4ba1; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73311911, jitterRate=0.09243260324001312}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:55:32,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7271d73f1b2300d37c9209b7cecf4ba1: Running coprocessor pre-open hook at 1675900470394Initializing all the Stores at 1675900532017 (+61623 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900532017Running coprocessor post-open hooks at 1675900532025 (+8 ms)Region opened successfully at 1675900532025 2023-02-08 23:55:32,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1., pid=94, masterSystemTime=1675900470384 2023-02-08 23:55:32,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1. 2023-02-08 23:55:32,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1. 2023-02-08 23:55:32,030 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=7271d73f1b2300d37c9209b7cecf4ba1, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:32,030 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900532030"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1675900532030"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1675900532030"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1675900532030"}]},"ts":"1675900532030"} 2023-02-08 23:55:32,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-08 23:55:32,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 7271d73f1b2300d37c9209b7cecf4ba1, server=jenkins-hbase4.apache.org,36713,1675900441074 in 1 mins, 1.802 sec 2023-02-08 23:55:32,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-08 23:55:32,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1, ASSIGN in 1 mins, 1.961 sec 2023-02-08 23:55:32,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.497 sec 2023-02-08 23:55:32,040 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1675900469536","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62504,"client":"172.31.14.131:37384","queuetimems":0,"class":"HMaster"} 2023-02-08 23:55:32,040 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-08 23:55:34,013 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up 2023-02-08 23:55:34,015 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:55:34,015 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,015 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,016 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,016 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,016 INFO [Listener at localhost/37973] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:34,017 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:34,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:34,019 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,019 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 23:55:34,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900534020"}]},"ts":"1675900534020"} 2023-02-08 23:55:34,021 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-08 23:55:34,024 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-08 23:55:34,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=4a9ac7b252881a6ce15fd79f0bb87322, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1, UNASSIGN}] 2023-02-08 23:55:34,025 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1, UNASSIGN 2023-02-08 23:55:34,026 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=7271d73f1b2300d37c9209b7cecf4ba1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:34,026 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900534025"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1675900534025"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675900534025"}]},"ts":"1675900534025"} 2023-02-08 23:55:34,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 7271d73f1b2300d37c9209b7cecf4ba1, server=jenkins-hbase4.apache.org,36713,1675900441074}] 2023-02-08 23:55:34,029 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=4a9ac7b252881a6ce15fd79f0bb87322, UNASSIGN 2023-02-08 23:55:34,029 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=4a9ac7b252881a6ce15fd79f0bb87322, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:34,029 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900534029"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900534029"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900534029"}]},"ts":"1675900534029"} 2023-02-08 23:55:34,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 4a9ac7b252881a6ce15fd79f0bb87322, server=jenkins-hbase4.apache.org,36199,1675900441120}] 2023-02-08 23:55:34,120 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,120 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 23:55:34,179 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7271d73f1b2300d37c9209b7cecf4ba1 2023-02-08 23:55:34,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7271d73f1b2300d37c9209b7cecf4ba1, disabling compactions & flushes 2023-02-08 23:55:34,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1. 2023-02-08 23:55:34,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1. 2023-02-08 23:55:34,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1. after waiting 0 ms 2023-02-08 23:55:34,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1. 2023-02-08 23:55:34,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:34,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1. 2023-02-08 23:55:34,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7271d73f1b2300d37c9209b7cecf4ba1: Waiting for close lock at 1675900534179Running coprocessor pre-close hooks at 1675900534179Disabling compacts and flushes for region at 1675900534179Disabling writes for close at 1675900534180 (+1 ms)Writing region close event to WAL at 1675900534180Running coprocessor post-close hooks at 1675900534180Closed at 1675900534181 (+1 ms) 2023-02-08 23:55:34,182 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:55:34,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4a9ac7b252881a6ce15fd79f0bb87322, disabling compactions & flushes 2023-02-08 23:55:34,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:55:34,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:55:34,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. after waiting 0 ms 2023-02-08 23:55:34,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:55:34,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7271d73f1b2300d37c9209b7cecf4ba1 2023-02-08 23:55:34,184 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=7271d73f1b2300d37c9209b7cecf4ba1, regionState=CLOSED 2023-02-08 23:55:34,185 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900534183"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1675900534183"}]},"ts":"1675900534183"} 2023-02-08 23:55:34,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-08 23:55:34,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-08 23:55:34,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 7271d73f1b2300d37c9209b7cecf4ba1, server=jenkins-hbase4.apache.org,36713,1675900441074 in 160 msec 2023-02-08 23:55:34,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:34,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322. 2023-02-08 23:55:34,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4a9ac7b252881a6ce15fd79f0bb87322: Waiting for close lock at 1675900534182Running coprocessor pre-close hooks at 1675900534182Disabling compacts and flushes for region at 1675900534182Disabling writes for close at 1675900534183 (+1 ms)Writing region close event to WAL at 1675900534183Running coprocessor post-close hooks at 1675900534189 (+6 ms)Closed at 1675900534190 (+1 ms) 2023-02-08 23:55:34,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1, UNASSIGN in 165 msec 2023-02-08 23:55:34,191 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:55:34,191 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=4a9ac7b252881a6ce15fd79f0bb87322, regionState=CLOSED 2023-02-08 23:55:34,191 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675900534191"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900534191"}]},"ts":"1675900534191"} 2023-02-08 23:55:34,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-08 23:55:34,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 4a9ac7b252881a6ce15fd79f0bb87322, server=jenkins-hbase4.apache.org,36199,1675900441120 in 162 msec 2023-02-08 23:55:34,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-08 23:55:34,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=4a9ac7b252881a6ce15fd79f0bb87322, UNASSIGN in 170 msec 2023-02-08 23:55:34,195 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900534195"}]},"ts":"1675900534195"} 2023-02-08 23:55:34,196 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-08 23:55:34,198 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-08 23:55:34,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 181 msec 2023-02-08 23:55:34,321 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,321 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 23:55:34,322 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-08 23:55:34,323 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,323 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,323 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:34,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:34,325 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:34,326 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,326 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,326 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:34,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-08 23:55:34,332 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:55:34,334 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/recovered.edits] 2023-02-08 23:55:34,339 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/recovered.edits/7.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322/recovered.edits/7.seqid 2023-02-08 23:55:34,340 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/4a9ac7b252881a6ce15fd79f0bb87322 2023-02-08 23:55:34,340 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-08 23:55:34,342 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:34,343 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-08 23:55:34,345 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-08 23:55:34,346 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:34,346 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-08 23:55:34,346 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900534346"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:34,346 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900534346"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:34,348 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-08 23:55:34,348 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4a9ac7b252881a6ce15fd79f0bb87322, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675900469028.4a9ac7b252881a6ce15fd79f0bb87322.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 7271d73f1b2300d37c9209b7cecf4ba1, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675900469028_0001.7271d73f1b2300d37c9209b7cecf4ba1.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-08 23:55:34,348 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-08 23:55:34,348 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675900534348"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:34,349 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-08 23:55:34,351 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:34,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 28 msec 2023-02-08 23:55:34,427 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,427 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-08 23:55:34,428 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-08 23:55:34,441 INFO [Listener at localhost/37973] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=344 (was 405), OpenFileDescriptor=651 (was 849), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=94 (was 254), ProcessCount=168 (was 168), AvailableMemoryMB=3103 (was 3296) 2023-02-08 23:55:34,453 INFO [Listener at localhost/37973] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=345, OpenFileDescriptor=651, MaxFileDescriptor=60000, SystemLoadAverage=94, ProcessCount=168, AvailableMemoryMB=3102 2023-02-08 23:55:34,453 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:55:34,453 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,453 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,455 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 23:55:34,456 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 23:55:34,457 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:34,458 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:55:34,458 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:55:34,460 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:34,460 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229 empty. 2023-02-08 23:55:34,461 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:34,461 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-08 23:55:34,475 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-08 23:55:34,476 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8333290d94b9c7a8ec4d6be5c844e229, NAME => 'testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp 2023-02-08 23:55:34,486 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:34,486 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 8333290d94b9c7a8ec4d6be5c844e229, disabling compactions & flushes 2023-02-08 23:55:34,486 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:34,486 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:34,486 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. after waiting 0 ms 2023-02-08 23:55:34,486 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:34,486 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:34,486 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 8333290d94b9c7a8ec4d6be5c844e229: Waiting for close lock at 1675900534486Disabling compacts and flushes for region at 1675900534486Disabling writes for close at 1675900534486Writing region close event to WAL at 1675900534486Closed at 1675900534486 2023-02-08 23:55:34,489 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:55:34,489 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675900534489"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900534489"}]},"ts":"1675900534489"} 2023-02-08 23:55:34,491 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:55:34,491 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:55:34,492 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900534491"}]},"ts":"1675900534491"} 2023-02-08 23:55:34,492 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-08 23:55:34,496 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:55:34,497 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:55:34,497 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:55:34,497 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:55:34,497 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 23:55:34,497 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=8333290d94b9c7a8ec4d6be5c844e229, ASSIGN}] 2023-02-08 23:55:34,499 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=8333290d94b9c7a8ec4d6be5c844e229, ASSIGN 2023-02-08 23:55:34,499 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=8333290d94b9c7a8ec4d6be5c844e229, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42343,1675900441151; forceNewPlan=false, retain=false 2023-02-08 23:55:34,649 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:55:34,650 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=8333290d94b9c7a8ec4d6be5c844e229, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:34,651 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675900534650"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900534650"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900534650"}]},"ts":"1675900534650"} 2023-02-08 23:55:34,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 8333290d94b9c7a8ec4d6be5c844e229, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:34,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:34,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8333290d94b9c7a8ec4d6be5c844e229, NAME => 'testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:55:34,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:55:34,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:34,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:34,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:34,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:34,809 INFO [StoreOpener-8333290d94b9c7a8ec4d6be5c844e229-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:34,810 DEBUG [StoreOpener-8333290d94b9c7a8ec4d6be5c844e229-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1 2023-02-08 23:55:34,810 DEBUG [StoreOpener-8333290d94b9c7a8ec4d6be5c844e229-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1 2023-02-08 23:55:34,811 INFO [StoreOpener-8333290d94b9c7a8ec4d6be5c844e229-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8333290d94b9c7a8ec4d6be5c844e229 columnFamilyName cf1 2023-02-08 23:55:34,811 INFO [StoreOpener-8333290d94b9c7a8ec4d6be5c844e229-1] regionserver.HStore(310): Store=8333290d94b9c7a8ec4d6be5c844e229/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:34,811 INFO [StoreOpener-8333290d94b9c7a8ec4d6be5c844e229-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:34,813 DEBUG [StoreOpener-8333290d94b9c7a8ec4d6be5c844e229-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2 2023-02-08 23:55:34,813 DEBUG [StoreOpener-8333290d94b9c7a8ec4d6be5c844e229-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2 2023-02-08 23:55:34,813 INFO [StoreOpener-8333290d94b9c7a8ec4d6be5c844e229-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8333290d94b9c7a8ec4d6be5c844e229 columnFamilyName cf2 2023-02-08 23:55:34,813 INFO [StoreOpener-8333290d94b9c7a8ec4d6be5c844e229-1] regionserver.HStore(310): Store=8333290d94b9c7a8ec4d6be5c844e229/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:34,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:34,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:34,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:34,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:34,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:55:34,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8333290d94b9c7a8ec4d6be5c844e229; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66475234, jitterRate=-0.00944182276725769}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:34,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8333290d94b9c7a8ec4d6be5c844e229: Running coprocessor pre-open hook at 1675900534808Writing region info on filesystem at 1675900534808Initializing all the Stores at 1675900534809 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900534809Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900534809Cleaning up temporary data from old regions at 1675900534815 (+6 ms)Cleaning up detritus from prior splits at 1675900534815Running coprocessor post-open hooks at 1675900534818 (+3 ms)Region opened successfully at 1675900534818 2023-02-08 23:55:34,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229., pid=104, masterSystemTime=1675900534803 2023-02-08 23:55:34,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:34,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:34,820 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=8333290d94b9c7a8ec4d6be5c844e229, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:34,821 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675900534820"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900534820"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900534820"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900534820"}]},"ts":"1675900534820"} 2023-02-08 23:55:34,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-08 23:55:34,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 8333290d94b9c7a8ec4d6be5c844e229, server=jenkins-hbase4.apache.org,42343,1675900441151 in 170 msec 2023-02-08 23:55:34,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-08 23:55:34,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=8333290d94b9c7a8ec4d6be5c844e229, ASSIGN in 326 msec 2023-02-08 23:55:34,825 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:55:34,825 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900534825"}]},"ts":"1675900534825"} 2023-02-08 23:55:34,826 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-08 23:55:34,828 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:55:34,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 372 msec 2023-02-08 23:55:34,964 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,964 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,964 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:34,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 23:55:34,966 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 23:55:34,966 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:34,966 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 23:55:34,966 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 23:55:34,967 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:34,967 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:34,968 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:34,968 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:34,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:34,968 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:34,968 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:34,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:34,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:34,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:34,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:34,968 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:34,968 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:34,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:34,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:34,970 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:34,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:34,970 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:34,970 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:34,970 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:34,970 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:34,970 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:34,970 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:34,970 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:34,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:34,971 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:34,971 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:34,971 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:34,971 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:34,971 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:34,971 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. started... 2023-02-08 23:55:34,971 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:34,971 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:34,971 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:34,972 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8333290d94b9c7a8ec4d6be5c844e229 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 23:55:34,972 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:34,973 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:34,973 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:34,973 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:34,974 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:34,974 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:34,986 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/f6dc61d2e9ec4ca3bcb0a3df02d440ae 2023-02-08 23:55:34,997 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/f878ae299bd9476ba1ab3f522762ac85 2023-02-08 23:55:35,001 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/f6dc61d2e9ec4ca3bcb0a3df02d440ae as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/f6dc61d2e9ec4ca3bcb0a3df02d440ae 2023-02-08 23:55:35,005 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/f6dc61d2e9ec4ca3bcb0a3df02d440ae, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:55:35,005 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/f878ae299bd9476ba1ab3f522762ac85 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/f878ae299bd9476ba1ab3f522762ac85 2023-02-08 23:55:35,009 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/f878ae299bd9476ba1ab3f522762ac85, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:55:35,009 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 8333290d94b9c7a8ec4d6be5c844e229 in 37ms, sequenceid=6, compaction requested=false 2023-02-08 23:55:35,009 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-08 23:55:35,010 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8333290d94b9c7a8ec4d6be5c844e229: 2023-02-08 23:55:35,010 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:35,010 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:35,010 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:35,010 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,010 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,010 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,012 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,012 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,012 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,012 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 23:55:35,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@645f3c98[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:35,014 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 23:55:35,014 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,015 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,015 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,015 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,015 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,015 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,015 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,015 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,015 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:35,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,015 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,016 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,016 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,016 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:35,015 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:35,016 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:35,018 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,019 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:35,021 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 23:55:35,021 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 23:55:35,021 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:35,021 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:35,021 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 23:55:35,021 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:35,023 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,023 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,023 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,023 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,023 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,023 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,023 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,023 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,024 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,024 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,026 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,026 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,026 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,028 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,029 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,029 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,029 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,029 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 23:55:35,029 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:35,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,029 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,029 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,029 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,029 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,029 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,029 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:35,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,029 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,029 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:35,030 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,030 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,030 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,030 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,030 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,030 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,030 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,030 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,130 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:35,130 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:35,130 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:35,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:35,141 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:35,141 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:35,142 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:35,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:35,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 23:55:35,143 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 23:55:35,143 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,144 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 23:55:35,144 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 23:55:35,144 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,144 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,146 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,146 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,146 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,146 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,147 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,147 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:35,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,148 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,148 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,148 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,148 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,148 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,148 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,148 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,148 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,148 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,148 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,148 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:35,148 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:35,148 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,149 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,148 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,149 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,149 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,149 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,148 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:35,149 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. started... 2023-02-08 23:55:35,149 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8333290d94b9c7a8ec4d6be5c844e229 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:35,153 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,153 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,153 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,153 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,153 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,153 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,159 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/4b55c3b1626f41cd825cc4f00e014ebf 2023-02-08 23:55:35,169 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/5e95ac3c49b94f0484f1edaa63f2650a 2023-02-08 23:55:35,172 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/4b55c3b1626f41cd825cc4f00e014ebf as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/4b55c3b1626f41cd825cc4f00e014ebf 2023-02-08 23:55:35,176 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/4b55c3b1626f41cd825cc4f00e014ebf, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:55:35,177 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/5e95ac3c49b94f0484f1edaa63f2650a as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/5e95ac3c49b94f0484f1edaa63f2650a 2023-02-08 23:55:35,181 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/5e95ac3c49b94f0484f1edaa63f2650a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:55:35,182 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8333290d94b9c7a8ec4d6be5c844e229 in 33ms, sequenceid=19, compaction requested=false 2023-02-08 23:55:35,182 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8333290d94b9c7a8ec4d6be5c844e229: 2023-02-08 23:55:35,182 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:35,182 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:35,182 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:35,182 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,182 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,183 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,184 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,190 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,190 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,190 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 23:55:35,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@437cfeff[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:35,197 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 23:55:35,197 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,200 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,200 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,200 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,200 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,201 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,201 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,201 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,201 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,201 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:35,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,201 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:35,201 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:35,201 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,201 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,201 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:35,202 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,202 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,202 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,202 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,202 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,202 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,203 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,203 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,203 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,203 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:35,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 23:55:35,210 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 23:55:35,210 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:35,210 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:35,210 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 23:55:35,210 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:35,212 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,212 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,212 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,212 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,212 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,212 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,212 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,213 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,214 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,214 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,217 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,217 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,218 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,223 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,223 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,223 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,223 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,224 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,224 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,223 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,223 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,224 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,224 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,224 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,224 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,224 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,224 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,224 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,224 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,224 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,225 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 23:55:35,225 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:35,225 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,225 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:35,225 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:35,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,325 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:35,325 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:35,326 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:35,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:35,337 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:35,337 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:35,338 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:35,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:35,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 23:55:35,339 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 23:55:35,339 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,339 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 23:55:35,339 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 23:55:35,340 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,340 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,341 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,341 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,341 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,341 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,342 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,342 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,343 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,343 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,343 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,343 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:35,343 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:35,343 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. started... 2023-02-08 23:55:35,343 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8333290d94b9c7a8ec4d6be5c844e229 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:35,344 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,344 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,345 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,345 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,345 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,345 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,352 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/63314dd1b8574d30aa65839588e42429 2023-02-08 23:55:35,364 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/a1df776ee80443f4ae3628f6f51e8f21 2023-02-08 23:55:35,367 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/63314dd1b8574d30aa65839588e42429 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/63314dd1b8574d30aa65839588e42429 2023-02-08 23:55:35,371 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/63314dd1b8574d30aa65839588e42429, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:55:35,371 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/a1df776ee80443f4ae3628f6f51e8f21 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/a1df776ee80443f4ae3628f6f51e8f21 2023-02-08 23:55:35,375 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/a1df776ee80443f4ae3628f6f51e8f21, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:55:35,375 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8333290d94b9c7a8ec4d6be5c844e229 in 32ms, sequenceid=32, compaction requested=true 2023-02-08 23:55:35,376 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8333290d94b9c7a8ec4d6be5c844e229: 2023-02-08 23:55:35,376 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:35,376 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:35,376 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:35,376 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,376 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,376 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,377 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,377 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,378 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,378 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 23:55:35,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b5fb10e[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:35,379 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 23:55:35,379 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,381 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,381 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,381 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,382 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,382 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,382 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,382 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,382 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:35,382 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:35,382 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,382 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:35,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,382 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,382 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,382 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:35,383 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,383 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,383 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,383 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,383 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,383 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,384 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,384 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,384 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,384 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:35,387 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 23:55:35,387 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 23:55:35,387 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:35,387 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:35,387 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 23:55:35,387 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:35,389 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,389 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,389 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,389 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,389 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,389 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,389 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,389 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,390 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,390 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,392 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,392 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,392 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,400 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,400 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,400 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,400 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,400 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,400 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,400 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,400 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,400 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 23:55:35,400 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:35,400 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,400 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,400 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,401 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,401 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:35,401 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:35,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,401 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,401 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,401 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,401 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,401 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,401 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,501 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:35,501 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:35,501 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:35,502 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:35,515 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:35,515 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:35,516 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:35,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:35,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 23:55:35,517 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 23:55:35,517 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,518 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 23:55:35,518 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 23:55:35,518 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,518 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,519 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,519 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,519 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,519 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,520 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,520 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:35,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,521 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,521 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,521 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,521 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,521 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,521 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,521 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,521 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,521 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,521 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:35,521 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. started... 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,521 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8333290d94b9c7a8ec4d6be5c844e229 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,522 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,523 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,523 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,523 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,523 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,523 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,523 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,530 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/165036231a834711ac9bc22f13cfab73 2023-02-08 23:55:35,540 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/96d4d3b0f9bc479c9b7055107d17614f 2023-02-08 23:55:35,544 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/165036231a834711ac9bc22f13cfab73 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/165036231a834711ac9bc22f13cfab73 2023-02-08 23:55:35,548 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/165036231a834711ac9bc22f13cfab73, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:55:35,548 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/96d4d3b0f9bc479c9b7055107d17614f as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/96d4d3b0f9bc479c9b7055107d17614f 2023-02-08 23:55:35,552 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/96d4d3b0f9bc479c9b7055107d17614f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:55:35,552 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8333290d94b9c7a8ec4d6be5c844e229 in 31ms, sequenceid=45, compaction requested=true 2023-02-08 23:55:35,552 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8333290d94b9c7a8ec4d6be5c844e229: 2023-02-08 23:55:35,552 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,554 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,554 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,554 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,555 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 23:55:35,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1617d05[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:35,556 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 23:55:35,556 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,558 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,558 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,558 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,559 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,559 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,559 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:35,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,559 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,559 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:35,559 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,559 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,559 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:35,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,559 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,559 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,559 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:35,560 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,560 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,560 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,561 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,561 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,561 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,561 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,562 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,562 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,562 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:35,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 23:55:35,564 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 23:55:35,564 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:35,564 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:35,564 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 23:55:35,564 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:35,567 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,567 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,567 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,567 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,567 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,567 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,567 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,568 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,568 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,569 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,570 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,571 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,571 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,573 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,573 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,573 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,573 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,573 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,573 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 23:55:35,574 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:35,573 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,573 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,574 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,574 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,574 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,574 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,574 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,574 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,573 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,573 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,574 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:35,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,574 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:35,574 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,574 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,575 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,674 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:35,675 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:35,675 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:35,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:35,685 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:35,686 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:35,686 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:35,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:35,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 23:55:35,688 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 23:55:35,688 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,688 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 23:55:35,688 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 23:55:35,688 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,688 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,689 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,690 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,689 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,690 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,690 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:35,690 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,691 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,691 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,691 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,692 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,691 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,692 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,692 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,692 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,692 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,692 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,692 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,692 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,692 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:35,692 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:35,692 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:35,692 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. started... 2023-02-08 23:55:35,692 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,692 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8333290d94b9c7a8ec4d6be5c844e229 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:35,693 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,695 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,695 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,695 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,695 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,695 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,700 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/336e2b6f80a34cebbb1f661567ac5e6f 2023-02-08 23:55:35,710 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/7ee7525a3cbe4cedb65c9087390eea08 2023-02-08 23:55:35,714 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/336e2b6f80a34cebbb1f661567ac5e6f as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/336e2b6f80a34cebbb1f661567ac5e6f 2023-02-08 23:55:35,717 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/336e2b6f80a34cebbb1f661567ac5e6f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:55:35,718 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/7ee7525a3cbe4cedb65c9087390eea08 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/7ee7525a3cbe4cedb65c9087390eea08 2023-02-08 23:55:35,722 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/7ee7525a3cbe4cedb65c9087390eea08, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:55:35,722 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8333290d94b9c7a8ec4d6be5c844e229 in 30ms, sequenceid=58, compaction requested=true 2023-02-08 23:55:35,723 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8333290d94b9c7a8ec4d6be5c844e229: 2023-02-08 23:55:35,723 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:35,723 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:35,723 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:35,723 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,723 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,723 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,724 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,724 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,725 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,725 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 23:55:35,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c472a4d[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:35,726 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 23:55:35,726 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,727 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,727 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,727 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,728 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,728 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:35,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:35,733 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,733 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,733 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,733 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,736 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,736 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,736 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,736 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,736 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,736 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:35,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 23:55:35,739 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 23:55:35,739 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:35,739 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:35,739 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 23:55:35,739 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:35,741 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,741 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,741 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,741 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,741 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,741 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,741 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,741 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,742 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,743 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,744 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,745 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,745 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,748 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,749 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,748 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,749 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,749 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:35,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,749 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,750 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,750 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:35,750 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:35,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,750 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,850 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:35,850 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:35,850 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:35,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:35,861 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:35,861 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:35,862 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:35,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:35,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 23:55:35,863 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 23:55:35,863 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,863 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 23:55:35,863 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 23:55:35,864 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,864 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,865 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,865 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,865 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,865 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,866 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,866 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:35,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,866 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,866 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,866 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,866 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:35,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 23:55:35,867 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,867 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,867 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:35,867 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:35,867 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:35,867 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. started... 2023-02-08 23:55:35,868 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8333290d94b9c7a8ec4d6be5c844e229 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:35,869 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,869 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,869 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,869 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,870 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,870 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,876 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/9d071eaf9bfc46259c764c4314d3f5dc 2023-02-08 23:55:35,886 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/88a72b7a581a41b2bceb0a591160e984 2023-02-08 23:55:35,889 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/9d071eaf9bfc46259c764c4314d3f5dc as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/9d071eaf9bfc46259c764c4314d3f5dc 2023-02-08 23:55:35,893 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/9d071eaf9bfc46259c764c4314d3f5dc, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:55:35,894 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/88a72b7a581a41b2bceb0a591160e984 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/88a72b7a581a41b2bceb0a591160e984 2023-02-08 23:55:35,897 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/88a72b7a581a41b2bceb0a591160e984, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:55:35,898 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8333290d94b9c7a8ec4d6be5c844e229 in 30ms, sequenceid=71, compaction requested=true 2023-02-08 23:55:35,898 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8333290d94b9c7a8ec4d6be5c844e229: 2023-02-08 23:55:35,898 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:35,898 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:35,898 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:35,898 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,898 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 23:55:35,898 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 23:55:35,900 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,900 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,900 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,900 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 23:55:35,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3868a43c[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:35,902 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 23:55:35,902 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,903 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,903 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,903 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,903 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,903 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,903 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:35,903 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,903 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:35,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,904 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,904 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 23:55:35,904 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,904 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:35,904 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 23:55:35,904 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,905 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,907 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 23:55:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,907 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,907 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 23:55:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:35,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 23:55:35,910 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 23:55:35,910 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:35,910 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:35,910 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 23:55:35,910 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:35,911 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,911 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,911 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:35,911 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,911 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,911 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,912 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:35,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,912 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,912 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,913 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:35,914 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 23:55:35,914 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,915 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,917 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,917 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,917 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,917 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,917 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 23:55:35,917 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:35,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,917 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,917 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:35,917 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,918 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,918 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,918 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 23:55:35,918 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:35,918 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,917 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:35,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,918 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:35,918 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:35,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:35,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:35,918 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:35,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,918 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:35,918 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:35,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:35,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:35,918 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 23:55:35,918 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 23:55:36,018 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:36,019 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,019 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:36,026 INFO [Listener at localhost/37973] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:36,027 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.StoreUtils(129): cannot split hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/4b55c3b1626f41cd825cc4f00e014ebf because midkey is the same as first or last row 2023-02-08 23:55:36,027 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.StoreUtils(129): cannot split hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/5e95ac3c49b94f0484f1edaa63f2650a because midkey is the same as first or last row 2023-02-08 23:55:36,027 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.HRegion(2744): Flushing 8333290d94b9c7a8ec4d6be5c844e229 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 23:55:36,036 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/e9615f6c4fac4ffaa23ddb8d7d587f51 2023-02-08 23:55:36,050 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/617987844a394829a3dfc48e570c58a3 2023-02-08 23:55:36,054 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf1/e9615f6c4fac4ffaa23ddb8d7d587f51 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/e9615f6c4fac4ffaa23ddb8d7d587f51 2023-02-08 23:55:36,058 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/e9615f6c4fac4ffaa23ddb8d7d587f51, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:55:36,059 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/.tmp/cf2/617987844a394829a3dfc48e570c58a3 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/617987844a394829a3dfc48e570c58a3 2023-02-08 23:55:36,063 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/617987844a394829a3dfc48e570c58a3, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:55:36,064 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 8333290d94b9c7a8ec4d6be5c844e229 in 37ms, sequenceid=82, compaction requested=true 2023-02-08 23:55:36,065 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.HRegion(2445): Flush status journal for 8333290d94b9c7a8ec4d6be5c844e229: 2023-02-08 23:55:36,066 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.StoreUtils(129): cannot split hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/4b55c3b1626f41cd825cc4f00e014ebf because midkey is the same as first or last row 2023-02-08 23:55:36,066 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42343] regionserver.StoreUtils(129): cannot split hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/5e95ac3c49b94f0484f1edaa63f2650a because midkey is the same as first or last row 2023-02-08 23:55:36,066 DEBUG [Listener at localhost/37973] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:36,066 DEBUG [Listener at localhost/37973] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-08 23:55:36,067 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:55:36,067 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,067 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,068 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,068 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,069 INFO [Listener at localhost/37973] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-08 23:55:36,069 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-08 23:55:36,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:36,072 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,072 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 23:55:36,073 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900536073"}]},"ts":"1675900536073"} 2023-02-08 23:55:36,074 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-08 23:55:36,076 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-08 23:55:36,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=8333290d94b9c7a8ec4d6be5c844e229, UNASSIGN}] 2023-02-08 23:55:36,086 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=8333290d94b9c7a8ec4d6be5c844e229, UNASSIGN 2023-02-08 23:55:36,086 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=8333290d94b9c7a8ec4d6be5c844e229, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:36,086 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675900536086"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900536086"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900536086"}]},"ts":"1675900536086"} 2023-02-08 23:55:36,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 8333290d94b9c7a8ec4d6be5c844e229, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:36,173 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,173 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 23:55:36,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:36,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8333290d94b9c7a8ec4d6be5c844e229, disabling compactions & flushes 2023-02-08 23:55:36,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:36,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:36,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. after waiting 0 ms 2023-02-08 23:55:36,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:36,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 23:55:36,245 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:36,245 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229. 2023-02-08 23:55:36,245 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8333290d94b9c7a8ec4d6be5c844e229: Waiting for close lock at 1675900536240Running coprocessor pre-close hooks at 1675900536240Disabling compacts and flushes for region at 1675900536240Disabling writes for close at 1675900536240Writing region close event to WAL at 1675900536241 (+1 ms)Running coprocessor post-close hooks at 1675900536245 (+4 ms)Closed at 1675900536245 2023-02-08 23:55:36,246 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:36,246 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=8333290d94b9c7a8ec4d6be5c844e229, regionState=CLOSED 2023-02-08 23:55:36,246 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675900536246"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900536246"}]},"ts":"1675900536246"} 2023-02-08 23:55:36,248 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-08 23:55:36,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 8333290d94b9c7a8ec4d6be5c844e229, server=jenkins-hbase4.apache.org,42343,1675900441151 in 159 msec 2023-02-08 23:55:36,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-08 23:55:36,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=8333290d94b9c7a8ec4d6be5c844e229, UNASSIGN in 172 msec 2023-02-08 23:55:36,250 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900536250"}]},"ts":"1675900536250"} 2023-02-08 23:55:36,251 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-08 23:55:36,252 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-08 23:55:36,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 184 msec 2023-02-08 23:55:36,374 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,374 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 23:55:36,375 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-08 23:55:36,375 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,375 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,376 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-08 23:55:36,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:36,378 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:36,378 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,378 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,379 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:36,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-08 23:55:36,382 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:36,384 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/recovered.edits] 2023-02-08 23:55:36,388 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/165036231a834711ac9bc22f13cfab73 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/165036231a834711ac9bc22f13cfab73 2023-02-08 23:55:36,389 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/336e2b6f80a34cebbb1f661567ac5e6f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/336e2b6f80a34cebbb1f661567ac5e6f 2023-02-08 23:55:36,390 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/4b55c3b1626f41cd825cc4f00e014ebf to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/4b55c3b1626f41cd825cc4f00e014ebf 2023-02-08 23:55:36,391 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/63314dd1b8574d30aa65839588e42429 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/63314dd1b8574d30aa65839588e42429 2023-02-08 23:55:36,392 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/9d071eaf9bfc46259c764c4314d3f5dc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/9d071eaf9bfc46259c764c4314d3f5dc 2023-02-08 23:55:36,393 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/e9615f6c4fac4ffaa23ddb8d7d587f51 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/e9615f6c4fac4ffaa23ddb8d7d587f51 2023-02-08 23:55:36,394 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/f6dc61d2e9ec4ca3bcb0a3df02d440ae to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf1/f6dc61d2e9ec4ca3bcb0a3df02d440ae 2023-02-08 23:55:36,396 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/5e95ac3c49b94f0484f1edaa63f2650a to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/5e95ac3c49b94f0484f1edaa63f2650a 2023-02-08 23:55:36,397 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/617987844a394829a3dfc48e570c58a3 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/617987844a394829a3dfc48e570c58a3 2023-02-08 23:55:36,398 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/7ee7525a3cbe4cedb65c9087390eea08 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/7ee7525a3cbe4cedb65c9087390eea08 2023-02-08 23:55:36,399 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/88a72b7a581a41b2bceb0a591160e984 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/88a72b7a581a41b2bceb0a591160e984 2023-02-08 23:55:36,400 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/96d4d3b0f9bc479c9b7055107d17614f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/96d4d3b0f9bc479c9b7055107d17614f 2023-02-08 23:55:36,401 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/a1df776ee80443f4ae3628f6f51e8f21 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/a1df776ee80443f4ae3628f6f51e8f21 2023-02-08 23:55:36,401 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/f878ae299bd9476ba1ab3f522762ac85 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/cf2/f878ae299bd9476ba1ab3f522762ac85 2023-02-08 23:55:36,403 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/recovered.edits/85.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229/recovered.edits/85.seqid 2023-02-08 23:55:36,404 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testInvalidSplitKey/8333290d94b9c7a8ec4d6be5c844e229 2023-02-08 23:55:36,404 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-08 23:55:36,406 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:36,407 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-08 23:55:36,410 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-08 23:55:36,411 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:36,411 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-08 23:55:36,411 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900536411"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:36,412 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-08 23:55:36,412 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 8333290d94b9c7a8ec4d6be5c844e229, NAME => 'testInvalidSplitKey,,1675900534456.8333290d94b9c7a8ec4d6be5c844e229.', STARTKEY => '', ENDKEY => ''}] 2023-02-08 23:55:36,412 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-08 23:55:36,412 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675900536412"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:36,414 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-08 23:55:36,415 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:36,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 39 msec 2023-02-08 23:55:36,479 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,479 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-08 23:55:36,480 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-08 23:55:36,493 INFO [Listener at localhost/37973] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=364 (was 345) - Thread LEAK? -, OpenFileDescriptor=738 (was 651) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=94 (was 94), ProcessCount=168 (was 168), AvailableMemoryMB=3097 (was 3102) 2023-02-08 23:55:36,505 INFO [Listener at localhost/37973] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=364, OpenFileDescriptor=738, MaxFileDescriptor=60000, SystemLoadAverage=94, ProcessCount=168, AvailableMemoryMB=3096 2023-02-08 23:55:36,506 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:55:36,506 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,506 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,508 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 23:55:36,508 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 23:55:36,509 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:36,511 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:55:36,511 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:55:36,513 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:36,513 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04 empty. 2023-02-08 23:55:36,514 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:36,514 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-08 23:55:36,522 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-08 23:55:36,524 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => ca3eeec7d5e5bdf23a6f2e4cfc167e04, NAME => 'testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp 2023-02-08 23:55:36,539 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:36,539 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing ca3eeec7d5e5bdf23a6f2e4cfc167e04, disabling compactions & flushes 2023-02-08 23:55:36,539 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:36,539 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:36,539 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. after waiting 0 ms 2023-02-08 23:55:36,539 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:36,539 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:36,539 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for ca3eeec7d5e5bdf23a6f2e4cfc167e04: Waiting for close lock at 1675900536539Disabling compacts and flushes for region at 1675900536539Disabling writes for close at 1675900536539Writing region close event to WAL at 1675900536539Closed at 1675900536539 2023-02-08 23:55:36,541 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:55:36,545 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900536544"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900536544"}]},"ts":"1675900536544"} 2023-02-08 23:55:36,548 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:55:36,549 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:55:36,549 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900536549"}]},"ts":"1675900536549"} 2023-02-08 23:55:36,550 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-08 23:55:36,555 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:55:36,556 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:55:36,556 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:55:36,556 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:55:36,556 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 23:55:36,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, ASSIGN}] 2023-02-08 23:55:36,557 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, ASSIGN 2023-02-08 23:55:36,558 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42343,1675900441151; forceNewPlan=false, retain=false 2023-02-08 23:55:36,708 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:55:36,709 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:36,710 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900536709"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900536709"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900536709"}]},"ts":"1675900536709"} 2023-02-08 23:55:36,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:36,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:36,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ca3eeec7d5e5bdf23a6f2e4cfc167e04, NAME => 'testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:55:36,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:55:36,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:36,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:36,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:36,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:36,866 INFO [StoreOpener-ca3eeec7d5e5bdf23a6f2e4cfc167e04-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:36,868 DEBUG [StoreOpener-ca3eeec7d5e5bdf23a6f2e4cfc167e04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1 2023-02-08 23:55:36,868 DEBUG [StoreOpener-ca3eeec7d5e5bdf23a6f2e4cfc167e04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1 2023-02-08 23:55:36,868 INFO [StoreOpener-ca3eeec7d5e5bdf23a6f2e4cfc167e04-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ca3eeec7d5e5bdf23a6f2e4cfc167e04 columnFamilyName cf1 2023-02-08 23:55:36,869 INFO [StoreOpener-ca3eeec7d5e5bdf23a6f2e4cfc167e04-1] regionserver.HStore(310): Store=ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:36,869 INFO [StoreOpener-ca3eeec7d5e5bdf23a6f2e4cfc167e04-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:36,870 DEBUG [StoreOpener-ca3eeec7d5e5bdf23a6f2e4cfc167e04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2 2023-02-08 23:55:36,870 DEBUG [StoreOpener-ca3eeec7d5e5bdf23a6f2e4cfc167e04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2 2023-02-08 23:55:36,870 INFO [StoreOpener-ca3eeec7d5e5bdf23a6f2e4cfc167e04-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ca3eeec7d5e5bdf23a6f2e4cfc167e04 columnFamilyName cf2 2023-02-08 23:55:36,871 INFO [StoreOpener-ca3eeec7d5e5bdf23a6f2e4cfc167e04-1] regionserver.HStore(310): Store=ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:36,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:36,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:36,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:36,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:36,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:55:36,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ca3eeec7d5e5bdf23a6f2e4cfc167e04; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64131250, jitterRate=-0.044369906187057495}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:36,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ca3eeec7d5e5bdf23a6f2e4cfc167e04: Running coprocessor pre-open hook at 1675900536866Writing region info on filesystem at 1675900536866Initializing all the Stores at 1675900536866Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900536866Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900536866Cleaning up temporary data from old regions at 1675900536872 (+6 ms)Cleaning up detritus from prior splits at 1675900536872Running coprocessor post-open hooks at 1675900536875 (+3 ms)Region opened successfully at 1675900536875 2023-02-08 23:55:36,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04., pid=111, masterSystemTime=1675900536862 2023-02-08 23:55:36,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:36,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:36,878 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:36,878 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900536878"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900536878"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900536878"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900536878"}]},"ts":"1675900536878"} 2023-02-08 23:55:36,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-08 23:55:36,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 in 168 msec 2023-02-08 23:55:36,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-08 23:55:36,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, ASSIGN in 324 msec 2023-02-08 23:55:36,882 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:55:36,882 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900536882"}]},"ts":"1675900536882"} 2023-02-08 23:55:36,883 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-08 23:55:36,885 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:55:36,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 376 msec 2023-02-08 23:55:37,016 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,016 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,016 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:37,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 23:55:37,018 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:37,018 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,018 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 23:55:37,018 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 23:55:37,018 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,018 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,020 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,020 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,020 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,020 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,021 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,021 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:37,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,022 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,022 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,022 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:37,022 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:37,023 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:37,023 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. started... 2023-02-08 23:55:37,024 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 23:55:37,024 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,025 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,025 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,025 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,025 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,025 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,032 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/9b65b4f96314417fa8ae579ee2b0d748 2023-02-08 23:55:37,044 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/24820c13435040239cbdd81ac0da0dde 2023-02-08 23:55:37,047 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/9b65b4f96314417fa8ae579ee2b0d748 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748 2023-02-08 23:55:37,051 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:55:37,052 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/24820c13435040239cbdd81ac0da0dde as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde 2023-02-08 23:55:37,055 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:55:37,056 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ca3eeec7d5e5bdf23a6f2e4cfc167e04 in 32ms, sequenceid=6, compaction requested=false 2023-02-08 23:55:37,056 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:37,057 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ca3eeec7d5e5bdf23a6f2e4cfc167e04: 2023-02-08 23:55:37,057 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:37,057 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:37,057 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:37,057 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,057 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,057 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,059 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,059 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,059 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,059 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,061 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 23:55:37,061 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ea9a5fe[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:37,061 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 23:55:37,061 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,063 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,063 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,063 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,064 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,064 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,064 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:37,064 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,064 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,064 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,064 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:37,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,064 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:37,064 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,064 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,064 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:37,065 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,065 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,065 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,065 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,065 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,065 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,065 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,066 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,066 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,066 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:37,070 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 23:55:37,070 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 23:55:37,070 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:37,070 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:37,070 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,070 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:37,071 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,071 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,072 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,072 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,073 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,073 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,075 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,075 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,075 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,079 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,079 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,079 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,079 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,080 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,080 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,079 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,079 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,080 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,080 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:37,080 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:37,080 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,080 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,080 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,080 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,080 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:37,080 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,080 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:37,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,080 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,081 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,081 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,081 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,181 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:37,181 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,181 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:37,193 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,193 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,194 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:37,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:37,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 23:55:37,196 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:37,196 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,196 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 23:55:37,196 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 23:55:37,196 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,196 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,198 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,198 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,198 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,198 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,198 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,198 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:37,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,199 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,199 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,199 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 23:55:37,199 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,199 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,199 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,199 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,202 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:37,203 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:37,203 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:37,203 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. started... 2023-02-08 23:55:37,203 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:37,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,204 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,204 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,204 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,204 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,205 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 23:55:37,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,205 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 23:55:37,205 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,205 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,205 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,205 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,205 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,205 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,205 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,205 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,205 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,205 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,207 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,207 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,207 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,207 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,217 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/59ee7b1f36104eb88a48694ac5c8296d 2023-02-08 23:55:37,235 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/08d96e436b5c4b8c9093b6092b4dde61 2023-02-08 23:55:37,240 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/59ee7b1f36104eb88a48694ac5c8296d as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d 2023-02-08 23:55:37,245 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:55:37,246 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/08d96e436b5c4b8c9093b6092b4dde61 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61 2023-02-08 23:55:37,250 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:55:37,251 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ca3eeec7d5e5bdf23a6f2e4cfc167e04 in 48ms, sequenceid=19, compaction requested=false 2023-02-08 23:55:37,251 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ca3eeec7d5e5bdf23a6f2e4cfc167e04: 2023-02-08 23:55:37,251 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:37,251 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:37,251 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:37,252 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,252 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,252 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,256 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,256 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,256 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,256 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 23:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24aa6c7b[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:37,259 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 23:55:37,260 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,261 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,261 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,261 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,261 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,261 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,261 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:37,261 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,261 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,261 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,261 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,261 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:37,261 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,262 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:37,261 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:37,263 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,263 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,263 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,264 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,264 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,264 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,264 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,264 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,264 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,264 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:37,272 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 23:55:37,272 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 23:55:37,272 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:37,272 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:37,272 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,272 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:37,275 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,275 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,275 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,275 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,275 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,275 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,276 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,276 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,277 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,277 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,279 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,280 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,280 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,283 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,283 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,283 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,283 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,283 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,283 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:37,283 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:37,283 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,283 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,283 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,284 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:37,284 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,284 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:37,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,284 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,284 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,284 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,284 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,284 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,284 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,284 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,285 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,285 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,384 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:37,384 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,384 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:37,397 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,397 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,397 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:37,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:37,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 23:55:37,399 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:37,399 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,399 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 23:55:37,399 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 23:55:37,399 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,399 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,402 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,402 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,402 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,402 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,403 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,403 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:37,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,404 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,404 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,404 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 23:55:37,404 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 23:55:37,404 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,404 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,405 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,405 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,405 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:37,405 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. started... 2023-02-08 23:55:37,405 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:37,406 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,406 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,407 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,407 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,407 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,407 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,416 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/2fdb224c0a5c47e48f86524f230039fd 2023-02-08 23:55:37,431 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/56d5576c0b34420795371943899eb63d 2023-02-08 23:55:37,435 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/2fdb224c0a5c47e48f86524f230039fd as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd 2023-02-08 23:55:37,439 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:55:37,440 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/56d5576c0b34420795371943899eb63d as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d 2023-02-08 23:55:37,445 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:55:37,446 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ca3eeec7d5e5bdf23a6f2e4cfc167e04 in 41ms, sequenceid=32, compaction requested=true 2023-02-08 23:55:37,446 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ca3eeec7d5e5bdf23a6f2e4cfc167e04: 2023-02-08 23:55:37,446 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:37,447 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:37,447 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:37,447 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,447 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,447 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,448 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,448 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,449 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,449 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,450 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 23:55:37,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19a3fe1[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:37,451 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 23:55:37,451 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,452 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,452 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,452 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,452 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,452 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:37,452 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,452 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,452 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:37,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,452 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,452 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,453 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:37,453 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,453 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,453 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:37,454 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,454 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,454 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,455 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,455 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,456 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,456 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,456 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,456 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,456 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:37,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 23:55:37,461 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 23:55:37,461 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:37,461 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:37,462 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,462 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:37,463 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,463 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,463 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,463 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,463 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,463 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,464 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,464 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,464 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,465 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,467 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,467 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,467 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,473 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,473 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,473 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,473 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,473 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,473 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,473 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:37,474 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:37,473 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,473 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,473 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,474 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,474 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,473 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,474 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,474 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:37,474 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,474 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,474 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:37,474 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,474 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,475 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,575 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:37,575 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,575 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:37,590 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,590 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,590 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:37,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:37,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 23:55:37,592 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:37,592 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,592 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 23:55:37,592 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 23:55:37,592 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,592 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,594 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,594 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,594 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,594 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,594 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,594 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:37,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,595 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,595 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,595 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,595 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,595 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,596 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 23:55:37,595 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,596 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:37,596 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,596 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. started... 2023-02-08 23:55:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,596 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,596 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:37,597 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,599 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,605 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/e8d46dd70a3240ce84b42e36a5faf717 2023-02-08 23:55:37,615 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/f7ba7d1a9b2444c1b510d919547eff2d 2023-02-08 23:55:37,619 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/e8d46dd70a3240ce84b42e36a5faf717 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717 2023-02-08 23:55:37,622 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:55:37,623 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/f7ba7d1a9b2444c1b510d919547eff2d as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d 2023-02-08 23:55:37,626 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:55:37,627 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ca3eeec7d5e5bdf23a6f2e4cfc167e04 in 31ms, sequenceid=45, compaction requested=true 2023-02-08 23:55:37,627 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ca3eeec7d5e5bdf23a6f2e4cfc167e04: 2023-02-08 23:55:37,627 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:37,627 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:37,627 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:37,627 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,627 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,627 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,630 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 23:55:37,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c2e32aa[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:37,632 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 23:55:37,632 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,634 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,634 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,634 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,634 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,634 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,634 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:37,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,634 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,634 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,634 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,634 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:37,634 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:37,634 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,634 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,634 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:37,635 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,636 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,635 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,636 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,636 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,636 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,636 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,636 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,636 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,636 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:37,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 23:55:37,640 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 23:55:37,640 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:37,640 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:37,640 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,640 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:37,643 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,643 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,643 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,643 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,643 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,643 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,643 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,643 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,644 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,644 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,646 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,646 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,646 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,670 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:37,671 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:37,670 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,670 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,671 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,671 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,671 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:37,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,671 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:37,671 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,671 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,672 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,672 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,771 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:37,772 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,772 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:37,782 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,782 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,782 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:37,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:37,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 23:55:37,784 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:37,784 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,784 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 23:55:37,784 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 23:55:37,784 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,784 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,786 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,786 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,786 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,786 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,786 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:37,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,787 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,787 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,787 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,787 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,787 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,787 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,788 ERROR [BP-379259138-172.31.14.131-1675900436765 heartbeating to localhost/127.0.0.1:38079] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-379259138-172.31.14.131-1675900436765 (Datanode Uuid e53af517-ccea-4786-a9f1-d1d57e9a860e) service to localhost/127.0.0.1:38079 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 23:55:37,788 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,788 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,788 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,788 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,788 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,788 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,788 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,788 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,788 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,788 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:37,788 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:37,788 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,788 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. started... 2023-02-08 23:55:37,788 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:37,788 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,788 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:37,788 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,789 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,789 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,789 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,789 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,789 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,792 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,792 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,792 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,793 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,793 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,796 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/3d9d315c61884930ae0f858b6483b912 2023-02-08 23:55:37,806 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/bf1e25331de544898c319348fee016a2 2023-02-08 23:55:37,809 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/3d9d315c61884930ae0f858b6483b912 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912 2023-02-08 23:55:37,812 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:55:37,813 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/bf1e25331de544898c319348fee016a2 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2 2023-02-08 23:55:37,816 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:55:37,817 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ca3eeec7d5e5bdf23a6f2e4cfc167e04 in 29ms, sequenceid=58, compaction requested=true 2023-02-08 23:55:37,817 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ca3eeec7d5e5bdf23a6f2e4cfc167e04: 2023-02-08 23:55:37,817 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:37,817 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:37,817 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:37,817 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,817 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,817 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,821 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,821 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,822 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,822 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,823 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 23:55:37,823 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f4bfe84[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:37,823 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 23:55:37,823 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,824 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,824 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,824 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,825 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,825 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,825 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:37,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,825 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,825 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,825 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,825 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:37,824 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,825 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,825 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:37,825 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:37,827 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,827 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,827 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,828 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,828 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,828 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,828 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,828 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,829 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,829 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:37,831 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 23:55:37,831 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 23:55:37,831 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:37,831 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:37,831 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,831 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:37,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,833 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,833 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,834 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,835 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,835 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,835 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,839 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,839 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:37,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,839 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:37,840 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:37,839 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,840 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,840 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,840 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,840 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:37,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:37,940 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:37,940 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,940 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:37,950 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,950 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,951 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:37,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:37,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 23:55:37,952 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:37,952 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,952 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 23:55:37,952 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 23:55:37,953 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,953 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,954 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,954 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,954 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:37,954 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:37,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:37,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,954 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,954 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,955 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,955 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,955 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,955 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 23:55:37,955 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,955 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 23:55:37,956 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,956 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,956 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,956 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:37,956 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:37,956 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:37,956 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 23:55:37,956 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. started... 2023-02-08 23:55:37,956 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,956 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,956 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,956 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 23:55:37,956 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:37,956 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 23:55:37,956 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:37,956 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 23:55:37,956 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,956 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,957 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,958 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,958 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,958 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,958 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,958 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,967 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/958f6123508c48df9585286422cd6935 2023-02-08 23:55:37,978 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/ed061e5440b64b8d919699c2db225c70 2023-02-08 23:55:37,982 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/958f6123508c48df9585286422cd6935 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935 2023-02-08 23:55:37,985 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:55:37,986 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/ed061e5440b64b8d919699c2db225c70 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70 2023-02-08 23:55:37,990 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:55:37,990 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ca3eeec7d5e5bdf23a6f2e4cfc167e04 in 34ms, sequenceid=71, compaction requested=true 2023-02-08 23:55:37,991 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ca3eeec7d5e5bdf23a6f2e4cfc167e04: 2023-02-08 23:55:37,991 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:37,991 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:37,991 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:37,991 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,991 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 23:55:37,991 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 23:55:37,993 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,993 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,993 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,993 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:37,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:37,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:37,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:37,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:37,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:37,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 23:55:37,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68f580a2[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:37,995 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 23:55:37,995 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,996 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,996 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,996 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,996 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,996 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,997 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:37,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,996 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:37,997 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,997 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 23:55:37,997 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,997 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:37,997 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:37,997 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 23:55:37,997 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:37,998 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,998 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,998 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,998 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,998 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,998 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,999 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,999 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 23:55:37,999 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:37,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:37,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:37,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:37,999 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 23:55:37,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:37,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:38,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:38,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:38,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:38,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:38,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:38,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:38,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:38,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:38,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:38,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:38,002 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 23:55:38,002 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 23:55:38,003 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:38,003 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:38,003 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 23:55:38,003 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:38,005 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,005 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,005 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:38,005 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,005 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:38,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:38,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:38,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:38,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:38,005 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,006 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:38,006 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:38,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:38,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:38,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:38,006 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:38,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:38,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:38,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:38,006 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:38,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:38,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:38,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:38,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:38,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:38,008 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:38,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:38,009 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 23:55:38,009 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:38,013 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:38,013 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:38,013 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:38,013 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:38,013 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:38,014 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:38,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:38,014 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:38,014 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 23:55:38,014 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:38,013 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:38,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:38,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:38,013 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:38,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:38,014 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:38,014 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:38,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:38,014 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:38,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:38,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:38,014 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:38,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:38,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:38,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:38,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:38,014 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,015 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:38,015 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,015 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:38,015 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:38,015 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,015 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 23:55:38,114 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:38,115 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:38,115 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:38,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:38,373 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-08 23:55:38,374 DEBUG [Listener at localhost/37973] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:38,375 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,376 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,376 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,376 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,377 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:55:38,377 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:55:38,377 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:55:38,625 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,625 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:55:38,625 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:55:38,625 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:55:38,625 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-08 23:55:38,626 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:55:38,626 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:55:38,626 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:55:38,626 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:55:38,626 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:55:38,628 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-08 23:55:38,628 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:55:38,628 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:55:38,628 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:55:38,628 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:55:38,629 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:38,629 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:55:38,629 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:38,629 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:38,629 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:38,629 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:55:38,629 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:38,630 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:38,630 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:38,630 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:55:38,630 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:55:38,630 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:38,630 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:38,631 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:38,631 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:55:38,631 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:38,631 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:38,631 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:38,631 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:55:38,631 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:38,631 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:38,632 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:38,632 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:38,632 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:38,632 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:38,633 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:38,633 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:38,633 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:38,633 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:38,633 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:38,633 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,633 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:55:38,633 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,633 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,634 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,634 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-08 23:55:38,634 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:55:38,635 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:55:38,635 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900568635 2023-02-08 23:55:38,636 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:55:38,637 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,639 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:55:38,639 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:55:38,639 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:55:38,639 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900598639 2023-02-08 23:55:38,639 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900658639 2023-02-08 23:55:38,639 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:55:38,640 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:55:38,642 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:55:38,642 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:55:38,642 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:55:38,890 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,890 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:55:38,890 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:55:38,890 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:55:38,890 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:55:38,890 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:55:38,890 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:55:38,890 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:55:38,890 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:55:38,891 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:55:38,892 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-08 23:55:38,892 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:55:38,893 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:55:38,893 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:55:38,893 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:55:38,893 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:38,893 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:55:38,894 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:38,894 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:38,894 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:38,894 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:55:38,894 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:38,894 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:38,894 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:38,895 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:55:38,895 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:55:38,895 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:38,895 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:38,896 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:38,896 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:55:38,896 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:38,896 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:38,896 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:38,897 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:55:38,897 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:38,897 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:38,897 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:38,897 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:38,898 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:38,898 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:38,898 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:38,898 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:38,898 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:38,898 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:38,898 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:38,899 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,899 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:55:38,899 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,899 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,899 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,899 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-08 23:55:38,899 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:55:38,900 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:55:38,900 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900568900 2023-02-08 23:55:38,902 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:55:38,902 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,905 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:55:38,905 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:55:38,905 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:55:38,905 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900598905 2023-02-08 23:55:38,905 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900658905 2023-02-08 23:55:38,905 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:55:38,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:55:38,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN}] 2023-02-08 23:55:38,907 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:55:38,908 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:38,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:55:38,908 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:55:39,156 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,156 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:55:39,156 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:55:39,156 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:55:39,156 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:55:39,156 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:55:39,156 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:55:39,156 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:55:39,156 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:55:39,157 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:55:39,158 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-08 23:55:39,158 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:55:39,159 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:55:39,159 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:55:39,159 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:55:39,159 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:39,159 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:55:39,159 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:39,159 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:39,160 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:39,160 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:55:39,160 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:39,160 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:39,160 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:39,160 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:55:39,161 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:55:39,161 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:39,161 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:39,161 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:39,161 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:55:39,161 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:39,161 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:39,161 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:39,162 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:55:39,162 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:39,162 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:39,162 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:39,162 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:39,162 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:39,163 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:39,163 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:39,163 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:39,163 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:39,163 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:39,163 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:39,163 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,163 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:55:39,164 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,164 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,164 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,164 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 23:55:39,164 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:55:39,165 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:55:39,165 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900569165 2023-02-08 23:55:39,167 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:55:39,167 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,169 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:55:39,169 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:55:39,169 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:55:39,169 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900599169 2023-02-08 23:55:39,169 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900659169 2023-02-08 23:55:39,169 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 23:55:39,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN}] 2023-02-08 23:55:39,169 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:55:39,171 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN 2023-02-08 23:55:39,171 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:39,172 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900539171"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900539171"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900539171"}]},"ts":"1675900539171"} 2023-02-08 23:55:39,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:39,324 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,324 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ca3eeec7d5e5bdf23a6f2e4cfc167e04, disabling compactions & flushes 2023-02-08 23:55:39,325 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:39,325 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:39,325 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. after waiting 0 ms 2023-02-08 23:55:39,325 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:39,325 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 23:55:39,334 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/d3e5280d698c4567b2236f940aa4847a 2023-02-08 23:55:39,345 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/94f78d66c9504dc9b0230e0a31733f58 2023-02-08 23:55:39,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf1/d3e5280d698c4567b2236f940aa4847a as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a 2023-02-08 23:55:39,353 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:55:39,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/.tmp/cf2/94f78d66c9504dc9b0230e0a31733f58 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58 2023-02-08 23:55:39,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:55:39,358 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ca3eeec7d5e5bdf23a6f2e4cfc167e04 in 33ms, sequenceid=82, compaction requested=true 2023-02-08 23:55:39,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 23:55:39,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:39,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. 2023-02-08 23:55:39,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ca3eeec7d5e5bdf23a6f2e4cfc167e04: Waiting for close lock at 1675900539324Running coprocessor pre-close hooks at 1675900539324Disabling compacts and flushes for region at 1675900539324Disabling writes for close at 1675900539325 (+1 ms)Obtaining lock to block concurrent updates at 1675900539325Preparing flush snapshotting stores in ca3eeec7d5e5bdf23a6f2e4cfc167e04 at 1675900539325Finished memstore snapshotting testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675900539325Flushing stores of testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. at 1675900539326 (+1 ms)Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1: creating writer at 1675900539326Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1: appending metadata at 1675900539328 (+2 ms)Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1: closing flushed file at 1675900539328Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2: creating writer at 1675900539338 (+10 ms)Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2: appending metadata at 1675900539340 (+2 ms)Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2: closing flushed file at 1675900539340Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1: reopening flushed file at 1675900539350 (+10 ms)Flushing ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2: reopening flushed file at 1675900539354 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ca3eeec7d5e5bdf23a6f2e4cfc167e04 in 33ms, sequenceid=82, compaction requested=true at 1675900539358 (+4 ms)Writing region close event to WAL at 1675900539360 (+2 ms)Running coprocessor post-close hooks at 1675900539363 (+3 ms)Closed at 1675900539364 (+1 ms) 2023-02-08 23:55:39,366 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,366 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=CLOSED 2023-02-08 23:55:39,366 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900539366"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900539366"}]},"ts":"1675900539366"} 2023-02-08 23:55:39,368 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-08 23:55:39,368 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 in 194 msec 2023-02-08 23:55:39,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-08 23:55:39,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN in 200 msec 2023-02-08 23:55:39,371 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:55:39,372 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:55:39,372 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:55:39,419 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,419 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:55:39,419 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:55:39,419 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:55:39,420 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:55:39,420 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:55:39,420 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:55:39,420 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:55:39,420 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:55:39,420 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:55:39,422 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 23:55:39,422 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:55:39,422 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:55:39,422 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:55:39,422 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:55:39,422 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:55:39,422 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:55:39,423 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:39,423 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:55:39,423 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:39,423 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:39,423 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:39,423 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:55:39,423 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:39,424 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:39,424 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:39,424 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:55:39,424 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:55:39,424 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:39,424 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:39,425 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:39,425 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:55:39,425 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:39,425 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:39,425 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:39,426 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:55:39,426 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:39,426 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:39,426 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:39,427 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:39,427 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:39,427 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:39,428 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:39,428 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:39,428 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:39,428 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:39,429 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:39,429 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,429 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN 2023-02-08 23:55:39,429 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:39,429 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:55:39,429 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,429 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,429 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,429 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:39,429 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:39,429 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-08 23:55:39,430 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:55:39,430 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:55:39,431 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900569431 2023-02-08 23:55:39,431 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:55:39,432 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,434 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:55:39,434 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:55:39,434 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:55:39,434 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900599434 2023-02-08 23:55:39,434 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900659434 2023-02-08 23:55:39,435 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:55:39,437 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:55:39,445 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, threads=14 2023-02-08 23:55:39,445 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,445 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,445 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,445 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,445 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,445 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,446 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,446 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,446 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,446 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,446 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,446 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,446 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,446 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,531 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,535 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,548 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,563 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,563 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,572 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,580 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,584 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,597 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,598 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,601 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,601 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,604 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,604 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,604 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region ca3eeec7d5e5bdf23a6f2e4cfc167e04 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 23:55:39,634 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:55:39,634 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:55:39,634 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:55:39,685 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,685 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:55:39,685 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:55:39,685 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:55:39,685 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:55:39,686 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:55:39,686 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:55:39,686 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:55:39,686 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:55:39,686 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:55:39,688 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-08 23:55:39,688 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:55:39,688 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:55:39,688 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:55:39,688 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:55:39,688 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:55:39,689 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:55:39,689 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:39,689 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:55:39,689 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:39,689 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:39,689 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:39,690 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:55:39,690 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:39,690 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:39,690 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:39,690 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:55:39,691 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:55:39,691 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:39,691 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:39,691 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:39,691 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:55:39,692 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:39,692 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:39,692 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:39,692 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:55:39,692 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:39,693 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:39,693 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:39,693 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:39,693 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:39,694 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:39,694 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:39,694 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:39,694 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:39,694 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:39,694 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:39,695 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,695 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN 2023-02-08 23:55:39,695 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:39,695 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:55:39,695 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,695 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,695 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,695 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:39,695 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:39,695 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-08 23:55:39,696 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:55:39,696 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:55:39,700 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900569700 2023-02-08 23:55:39,701 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:39,701 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:55:39,703 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:55:39,704 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:55:39,704 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:55:39,704 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900599704 2023-02-08 23:55:39,704 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900659704 2023-02-08 23:55:39,704 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 23:55:39,713 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, threads=14 2023-02-08 23:55:39,714 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,714 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,714 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,714 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,714 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,716 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,716 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,716 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,720 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,720 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,720 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,720 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,720 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,721 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,723 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:55:39,860 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,879 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,888 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,903 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,908 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,908 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,909 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,911 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,914 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,914 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,914 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,915 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:39,919 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912 for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:40,315 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a for region: ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:40,315 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region ca3eeec7d5e5bdf23a6f2e4cfc167e04 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 23:55:40,345 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:55:40,350 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 23:55:40,351 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 23:55:40,351 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:55:40,351 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,351 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:55:40,351 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:55:40,454 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,454 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:55:40,454 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:55:40,455 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:55:40,455 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:55:40,455 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:55:40,455 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:55:40,455 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:55:40,455 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:55:40,456 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:55:40,457 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-08 23:55:40,457 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:55:40,457 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:55:40,457 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:55:40,457 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:55:40,457 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:55:40,457 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:55:40,458 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:40,458 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:55:40,458 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:40,458 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:40,458 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:40,458 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:55:40,458 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:40,458 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:40,459 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:40,459 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:55:40,459 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:55:40,459 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:40,459 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:40,459 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:40,459 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:55:40,459 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:40,460 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:40,460 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:40,460 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:55:40,460 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:40,460 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:40,460 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:40,460 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:40,461 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:40,461 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:40,461 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:40,461 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:40,461 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:40,462 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:40,462 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:40,462 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,462 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN 2023-02-08 23:55:40,462 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:40,463 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:55:40,463 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,463 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,463 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,463 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:40,463 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:40,463 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-08 23:55:40,464 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:55:40,464 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:55:40,466 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900570466 2023-02-08 23:55:40,468 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:55:40,468 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,470 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:55:40,470 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:55:40,470 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:55:40,470 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900600470 2023-02-08 23:55:40,470 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900660470 2023-02-08 23:55:40,470 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 23:55:40,475 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:55:40,477 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:55:40,477 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:55:40,477 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:55:40,721 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,721 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:55:40,721 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:55:40,721 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:55:40,721 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:55:40,721 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:55:40,721 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:55:40,721 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:55:40,721 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:55:40,722 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:55:40,723 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-08 23:55:40,723 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:55:40,723 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:55:40,723 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:55:40,724 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:55:40,724 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:55:40,724 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:55:40,724 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:40,724 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:55:40,724 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:40,724 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:40,725 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:40,725 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:55:40,725 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:40,725 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:40,725 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:40,725 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:55:40,725 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:55:40,725 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:40,726 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:40,726 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:40,726 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:55:40,726 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:40,726 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:40,726 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:40,726 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:55:40,726 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:40,726 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:40,727 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:40,727 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:40,727 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:40,727 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:40,727 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:40,727 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:40,727 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:40,728 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:40,728 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:40,728 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,728 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN 2023-02-08 23:55:40,728 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:40,728 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:55:40,728 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,728 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,728 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,728 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:40,729 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:40,729 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 23:55:40,729 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:55:40,730 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:55:40,730 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900570730 2023-02-08 23:55:40,730 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:55:40,732 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,734 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:55:40,734 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:55:40,734 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:55:40,734 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900600734 2023-02-08 23:55:40,734 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900660734 2023-02-08 23:55:40,734 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 23:55:40,734 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:55:40,736 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900540736"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675900540736"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675900540736"}]},"ts":"1675900540736"} 2023-02-08 23:55:40,736 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900540736"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900540736"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900540736"}]},"ts":"1675900540736"} 2023-02-08 23:55:40,737 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900540736"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900540736"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900540736"}]},"ts":"1675900540736"} 2023-02-08 23:55:40,738 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:55:40,738 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:55:40,738 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:55:40,787 ERROR [BP-379259138-172.31.14.131-1675900436765 heartbeating to localhost/127.0.0.1:38079] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-379259138-172.31.14.131-1675900436765 (Datanode Uuid d39c9230-801b-44ca-bfae-38ddf4db7d5a) service to localhost/127.0.0.1:38079 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 23:55:40,984 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,984 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:55:40,985 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:55:40,985 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:55:40,985 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:55:40,985 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:55:40,985 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:55:40,985 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:55:40,985 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:55:40,986 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:55:40,987 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-08 23:55:40,987 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:55:40,987 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:55:40,987 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:55:40,988 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:55:40,988 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:55:40,988 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:55:40,988 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:40,989 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:55:40,989 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:40,989 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:40,989 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:40,990 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:55:40,990 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:40,990 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:40,990 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:40,991 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:55:40,991 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:55:40,991 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:40,991 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:40,991 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:40,991 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:55:40,991 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:40,992 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:40,992 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:40,992 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:55:40,992 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:40,992 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:40,992 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:40,993 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:40,993 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:40,993 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:40,993 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:40,993 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:40,994 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:40,994 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:40,994 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:40,994 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,994 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN 2023-02-08 23:55:40,994 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:40,994 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:55:40,994 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,994 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,994 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,994 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:40,995 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:40,995 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-08 23:55:40,995 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:55:40,996 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:55:40,996 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900570996 2023-02-08 23:55:40,996 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:55:40,997 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:40,999 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:55:40,999 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:55:41,000 INFO [Listener at localhost/37973] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,42343,1675900441151, table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:41,000 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=add5f994245cd9a6365fc1ff87be30da, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 23:55:41,000 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8a49dbb5e04b34d3140f2408d1509844, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 23:55:41,000 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:55:41,000 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900601000 2023-02-08 23:55:41,000 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900661000 2023-02-08 23:55:41,000 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 23:55:41,001 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900541000"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675900541000"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675900541000"}]},"ts":"1675900541000"} 2023-02-08 23:55:41,001 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900541000"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900541000"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900541000"}]},"ts":"1675900541000"} 2023-02-08 23:55:41,001 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900541000"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900541000"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900541000"}]},"ts":"1675900541000"} 2023-02-08 23:55:41,002 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:55:41,004 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:55:41,004 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:55:41,004 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:55:41,250 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,250 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:55:41,251 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:55:41,251 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:55:41,251 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:55:41,251 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:55:41,251 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:55:41,251 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:55:41,251 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:55:41,252 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:55:41,253 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-08 23:55:41,253 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:55:41,253 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:55:41,253 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:55:41,254 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:55:41,254 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:55:41,254 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:55:41,254 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:41,255 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:55:41,255 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:41,255 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:41,255 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:41,255 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:55:41,255 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:41,256 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:41,256 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:41,256 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:55:41,256 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:55:41,256 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:41,256 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:41,257 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:41,257 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:55:41,257 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:41,257 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:41,257 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:41,257 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:55:41,257 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:41,257 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:41,258 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:41,258 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:41,258 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:41,258 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:41,258 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:41,258 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:41,258 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:41,259 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:41,259 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:41,259 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,259 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN 2023-02-08 23:55:41,259 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:41,259 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:55:41,259 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,259 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,259 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,259 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:41,259 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:41,260 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-08 23:55:41,260 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:55:41,260 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:55:41,261 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900571261 2023-02-08 23:55:41,261 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:55:41,262 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,264 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:55:41,264 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:55:41,264 INFO [Listener at localhost/37973] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,42343,1675900441151, table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:41,265 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=add5f994245cd9a6365fc1ff87be30da, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 23:55:41,265 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8a49dbb5e04b34d3140f2408d1509844, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 23:55:41,265 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:55:41,265 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900601265 2023-02-08 23:55:41,265 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900661265 2023-02-08 23:55:41,265 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 23:55:41,265 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:55:41,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=add5f994245cd9a6365fc1ff87be30da, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a49dbb5e04b34d3140f2408d1509844, ASSIGN}] 2023-02-08 23:55:41,267 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:55:41,267 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:55:41,267 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:55:41,515 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,515 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:55:41,515 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:55:41,515 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:55:41,516 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:55:41,516 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:55:41,516 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:55:41,516 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:55:41,516 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:55:41,517 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:55:41,518 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-08 23:55:41,518 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:55:41,518 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:55:41,518 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:55:41,519 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:55:41,519 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:55:41,519 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:55:41,519 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:41,520 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:55:41,520 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:41,520 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:41,520 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:41,521 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:55:41,521 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:41,521 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:41,521 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:41,521 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:55:41,522 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:55:41,522 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:41,522 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:41,522 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:41,522 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:55:41,523 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:41,523 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:41,523 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:41,523 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:55:41,523 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:41,523 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:41,524 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:41,524 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:41,524 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:41,524 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:41,525 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:41,525 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:41,525 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:41,525 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:41,525 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:41,525 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,526 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN 2023-02-08 23:55:41,526 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:41,526 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:55:41,526 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,526 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,526 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,526 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:41,526 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:41,526 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-08 23:55:41,527 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:55:41,527 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:55:41,527 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900571527 2023-02-08 23:55:41,528 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:55:41,529 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,531 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:55:41,531 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:55:41,531 INFO [Listener at localhost/37973] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,42343,1675900441151, table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:41,531 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=add5f994245cd9a6365fc1ff87be30da, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 23:55:41,531 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8a49dbb5e04b34d3140f2408d1509844, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 23:55:41,532 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:55:41,532 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900601532 2023-02-08 23:55:41,532 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900661532 2023-02-08 23:55:41,532 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 23:55:41,532 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=add5f994245cd9a6365fc1ff87be30da, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a49dbb5e04b34d3140f2408d1509844, ASSIGN}] 2023-02-08 23:55:41,532 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:55:41,534 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a49dbb5e04b34d3140f2408d1509844, ASSIGN 2023-02-08 23:55:41,534 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a49dbb5e04b34d3140f2408d1509844, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,42343,1675900441151; forceNewPlan=false, retain=false 2023-02-08 23:55:41,535 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=add5f994245cd9a6365fc1ff87be30da, ASSIGN 2023-02-08 23:55:41,535 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=add5f994245cd9a6365fc1ff87be30da, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,42343,1675900441151; forceNewPlan=false, retain=false 2023-02-08 23:55:41,684 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 23:55:41,686 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=8a49dbb5e04b34d3140f2408d1509844, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:41,686 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900541686"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900541686"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900541686"}]},"ts":"1675900541686"} 2023-02-08 23:55:41,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure 8a49dbb5e04b34d3140f2408d1509844, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:41,689 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=add5f994245cd9a6365fc1ff87be30da, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:41,689 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900541689"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900541689"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900541689"}]},"ts":"1675900541689"} 2023-02-08 23:55:41,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure add5f994245cd9a6365fc1ff87be30da, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:41,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da. 2023-02-08 23:55:41,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => add5f994245cd9a6365fc1ff87be30da, NAME => 'testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.', STARTKEY => '', ENDKEY => '41'} 2023-02-08 23:55:41,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:55:41,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution add5f994245cd9a6365fc1ff87be30da 2023-02-08 23:55:41,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:41,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for add5f994245cd9a6365fc1ff87be30da 2023-02-08 23:55:41,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for add5f994245cd9a6365fc1ff87be30da 2023-02-08 23:55:41,843 INFO [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region add5f994245cd9a6365fc1ff87be30da 2023-02-08 23:55:41,844 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1 2023-02-08 23:55:41,845 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1 2023-02-08 23:55:41,845 INFO [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region add5f994245cd9a6365fc1ff87be30da columnFamilyName cf1 2023-02-08 23:55:41,857 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd-bottom 2023-02-08 23:55:41,860 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912-bottom 2023-02-08 23:55:41,864 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d-bottom 2023-02-08 23:55:41,867 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935-bottom 2023-02-08 23:55:41,870 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748-bottom 2023-02-08 23:55:41,873 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a-bottom 2023-02-08 23:55:41,877 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717-bottom 2023-02-08 23:55:41,877 INFO [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(310): Store=add5f994245cd9a6365fc1ff87be30da/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:41,877 INFO [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region add5f994245cd9a6365fc1ff87be30da 2023-02-08 23:55:41,878 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2 2023-02-08 23:55:41,878 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2 2023-02-08 23:55:41,878 INFO [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region add5f994245cd9a6365fc1ff87be30da columnFamilyName cf2 2023-02-08 23:55:41,889 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61-bottom 2023-02-08 23:55:41,892 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde-bottom 2023-02-08 23:55:41,895 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d-bottom 2023-02-08 23:55:41,899 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58-bottom 2023-02-08 23:55:41,902 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2-bottom 2023-02-08 23:55:41,906 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70-bottom 2023-02-08 23:55:41,910 DEBUG [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d-bottom 2023-02-08 23:55:41,910 INFO [StoreOpener-add5f994245cd9a6365fc1ff87be30da-1] regionserver.HStore(310): Store=add5f994245cd9a6365fc1ff87be30da/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:41,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da 2023-02-08 23:55:41,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da 2023-02-08 23:55:41,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:41,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for add5f994245cd9a6365fc1ff87be30da 2023-02-08 23:55:41,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened add5f994245cd9a6365fc1ff87be30da; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71833886, jitterRate=0.07040831446647644}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:41,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for add5f994245cd9a6365fc1ff87be30da: Running coprocessor pre-open hook at 1675900541842Writing region info on filesystem at 1675900541842Initializing all the Stores at 1675900541843 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900541843Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900541843Cleaning up temporary data from old regions at 1675900541913 (+70 ms)Cleaning up detritus from prior splits at 1675900541913Running coprocessor post-open hooks at 1675900541916 (+3 ms)Region opened successfully at 1675900541916 2023-02-08 23:55:41,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da., pid=118, masterSystemTime=1675900541839 2023-02-08 23:55:41,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:41,919 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:55:41,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:41,919 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da. 2023-02-08 23:55:41,919 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:55:41,919 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1912): add5f994245cd9a6365fc1ff87be30da/cf1 is initiating minor compaction (all files) 2023-02-08 23:55:41,920 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da. 2023-02-08 23:55:41,920 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1912): add5f994245cd9a6365fc1ff87be30da/cf2 is initiating minor compaction (all files) 2023-02-08 23:55:41,920 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of add5f994245cd9a6365fc1ff87be30da/cf1 in testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da. 2023-02-08 23:55:41,920 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2258): Starting compaction of add5f994245cd9a6365fc1ff87be30da/cf2 in testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da. 2023-02-08 23:55:41,920 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58-bottom] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/.tmp, totalSize=34.9 K 2023-02-08 23:55:41,920 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a-bottom] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/.tmp, totalSize=34.9 K 2023-02-08 23:55:41,921 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900537012 2023-02-08 23:55:41,921 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900537012 2023-02-08 23:55:41,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da. 2023-02-08 23:55:41,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da. 2023-02-08 23:55:41,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844. 2023-02-08 23:55:41,921 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900537183 2023-02-08 23:55:41,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8a49dbb5e04b34d3140f2408d1509844, NAME => 'testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.', STARTKEY => '41', ENDKEY => ''} 2023-02-08 23:55:41,921 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900537183 2023-02-08 23:55:41,921 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=add5f994245cd9a6365fc1ff87be30da, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:41,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:55:41,922 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900541921"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900541921"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900541921"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900541921"}]},"ts":"1675900541921"} 2023-02-08 23:55:41,922 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900537386 2023-02-08 23:55:41,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,922 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900537386 2023-02-08 23:55:41,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:41,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,922 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900537576 2023-02-08 23:55:41,922 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900537576 2023-02-08 23:55:41,923 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900537773 2023-02-08 23:55:41,923 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900537941 2023-02-08 23:55:41,923 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900537773 2023-02-08 23:55:41,924 INFO [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,924 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675900538116 2023-02-08 23:55:41,924 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900537941 2023-02-08 23:55:41,925 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675900538116 2023-02-08 23:55:41,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-02-08 23:55:41,926 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1 2023-02-08 23:55:41,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure add5f994245cd9a6365fc1ff87be30da, server=jenkins-hbase4.apache.org,42343,1675900441151 in 233 msec 2023-02-08 23:55:41,926 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1 2023-02-08 23:55:41,926 INFO [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8a49dbb5e04b34d3140f2408d1509844 columnFamilyName cf1 2023-02-08 23:55:41,927 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=add5f994245cd9a6365fc1ff87be30da, ASSIGN in 394 msec 2023-02-08 23:55:41,938 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] throttle.PressureAwareThroughputController(145): add5f994245cd9a6365fc1ff87be30da#cf1#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:41,940 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] throttle.PressureAwareThroughputController(145): add5f994245cd9a6365fc1ff87be30da#cf2#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:41,941 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd-top 2023-02-08 23:55:41,944 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912-top 2023-02-08 23:55:41,946 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d-top 2023-02-08 23:55:41,949 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935-top 2023-02-08 23:55:41,960 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748-top 2023-02-08 23:55:41,964 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a-top 2023-02-08 23:55:41,965 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/.tmp/cf2/58b4d818a02b4bf48a2017a22e9065bc as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/58b4d818a02b4bf48a2017a22e9065bc 2023-02-08 23:55:41,970 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717-top 2023-02-08 23:55:41,971 INFO [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(310): Store=8a49dbb5e04b34d3140f2408d1509844/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:41,971 INFO [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,972 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2 2023-02-08 23:55:41,972 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2 2023-02-08 23:55:41,972 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in add5f994245cd9a6365fc1ff87be30da/cf2 of add5f994245cd9a6365fc1ff87be30da into 58b4d818a02b4bf48a2017a22e9065bc(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:41,972 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for add5f994245cd9a6365fc1ff87be30da: 2023-02-08 23:55:41,972 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da., storeName=add5f994245cd9a6365fc1ff87be30da/cf2, priority=9, startTime=1675900541919; duration=0sec 2023-02-08 23:55:41,972 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:41,972 INFO [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8a49dbb5e04b34d3140f2408d1509844 columnFamilyName cf2 2023-02-08 23:55:41,982 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61-top 2023-02-08 23:55:41,984 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde-top 2023-02-08 23:55:41,987 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d-top 2023-02-08 23:55:41,989 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58-top 2023-02-08 23:55:41,992 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2-top 2023-02-08 23:55:41,994 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70-top 2023-02-08 23:55:41,998 DEBUG [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d-top 2023-02-08 23:55:41,998 INFO [StoreOpener-8a49dbb5e04b34d3140f2408d1509844-1] regionserver.HStore(310): Store=8a49dbb5e04b34d3140f2408d1509844/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:41,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:41,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:42,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8a49dbb5e04b34d3140f2408d1509844; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62458632, jitterRate=-0.06929385662078857}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:42,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8a49dbb5e04b34d3140f2408d1509844: Running coprocessor pre-open hook at 1675900541922Writing region info on filesystem at 1675900541922Initializing all the Stores at 1675900541923 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900541923Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900541924 (+1 ms)Cleaning up temporary data from old regions at 1675900542000 (+76 ms)Cleaning up detritus from prior splits at 1675900542000Running coprocessor post-open hooks at 1675900542003 (+3 ms)Region opened successfully at 1675900542003 2023-02-08 23:55:42,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844., pid=117, masterSystemTime=1675900541839 2023-02-08 23:55:42,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:55:42,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:55:42,005 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:55:42,005 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844. 2023-02-08 23:55:42,005 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1912): 8a49dbb5e04b34d3140f2408d1509844/cf1 is initiating minor compaction (all files) 2023-02-08 23:55:42,005 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8a49dbb5e04b34d3140f2408d1509844/cf1 in testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844. 2023-02-08 23:55:42,006 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a-top] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/.tmp, totalSize=34.9 K 2023-02-08 23:55:42,006 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675900537012 2023-02-08 23:55:42,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844. 2023-02-08 23:55:42,006 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675900537183 2023-02-08 23:55:42,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844. 2023-02-08 23:55:42,007 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=8a49dbb5e04b34d3140f2408d1509844, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:42,007 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675900537386 2023-02-08 23:55:42,007 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900542007"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900542007"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900542007"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900542007"}]},"ts":"1675900542007"} 2023-02-08 23:55:42,007 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675900537576 2023-02-08 23:55:42,008 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675900537773 2023-02-08 23:55:42,008 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675900537941 2023-02-08 23:55:42,009 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675900538116 2023-02-08 23:55:42,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-08 23:55:42,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 8a49dbb5e04b34d3140f2408d1509844, server=jenkins-hbase4.apache.org,42343,1675900441151 in 321 msec 2023-02-08 23:55:42,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-02-08 23:55:42,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a49dbb5e04b34d3140f2408d1509844, ASSIGN in 478 msec 2023-02-08 23:55:42,011 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 23:55:42,011 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:55:42,012 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 23:55:42,018 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] throttle.PressureAwareThroughputController(145): 8a49dbb5e04b34d3140f2408d1509844#cf1#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:42,030 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/.tmp/cf1/0986e7e15b3e40559aafc8d28fe075f0 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/0986e7e15b3e40559aafc8d28fe075f0 2023-02-08 23:55:42,032 INFO [Listener at localhost/37973] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,032 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 23:55:42,032 INFO [Listener at localhost/37973] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:55:42,032 INFO [Listener at localhost/37973] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:55:42,032 ERROR [Listener at localhost/37973] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 23:55:42,032 INFO [Listener at localhost/37973] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 23:55:42,032 INFO [Listener at localhost/37973] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 23:55:42,033 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 23:55:42,033 INFO [Listener at localhost/37973] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:55:42,033 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 23:55:42,034 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-08 23:55:42,034 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8a49dbb5e04b34d3140f2408d1509844/cf1 of 8a49dbb5e04b34d3140f2408d1509844 into 0986e7e15b3e40559aafc8d28fe075f0(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:42,034 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 23:55:42,034 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8a49dbb5e04b34d3140f2408d1509844: 2023-02-08 23:55:42,034 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 23:55:42,034 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844., storeName=8a49dbb5e04b34d3140f2408d1509844/cf1, priority=9, startTime=1675900542004; duration=0sec 2023-02-08 23:55:42,035 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 21] 2023-02-08 23:55:42,035 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 20] 2023-02-08 23:55:42,035 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:55:42,035 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 23] 2023-02-08 23:55:42,035 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:55:42,035 DEBUG [Listener at localhost/37973] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 22] 2023-02-08 23:55:42,035 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844. 2023-02-08 23:55:42,035 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1912): 8a49dbb5e04b34d3140f2408d1509844/cf2 is initiating minor compaction (all files) 2023-02-08 23:55:42,035 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8a49dbb5e04b34d3140f2408d1509844/cf2 in testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844. 2023-02-08 23:55:42,035 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 23:55:42,035 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58-top] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/.tmp, totalSize=34.9 K 2023-02-08 23:55:42,035 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 23:55:42,035 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 23:55:42,036 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:55:42,036 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675900537012 2023-02-08 23:55:42,036 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:42,036 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=952532b0193ae630451f659ecaa3180f, daughterA=5981761d0d9fef4c60f7e5990070c751, daughterB=35b9c4ae42935ae148dc156df868a3bc 2023-02-08 23:55:42,036 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:42,036 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 23:55:42,036 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:42,037 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=af07d15277922930e96dbc9b857cb0df, daughterA=fe03d2fe0b831458763c25fce0549d32, daughterB=335bec642c9991062859430c97058271 2023-02-08 23:55:42,037 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:42,037 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 23:55:42,037 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:42,037 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=8ef507e9697a1fef706cfdf1625d1596, daughterA=0a57f3ef37544e34b96c0446268af1c5, daughterB=1a75a4d55dc470081989e63e23308c9b 2023-02-08 23:55:42,037 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=8ef507e9697a1fef706cfdf1625d1596, ASSIGN 2023-02-08 23:55:42,037 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:42,037 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 23:55:42,037 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:42,038 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a2b657ee6899d54a60f2b2b3f0c2f7fd, daughterA=2257771f8158addd99a8446b310dd3fe, daughterB=0a89e703978b35555b1c9da5e800b1a9 2023-02-08 23:55:42,038 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:42,038 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 23:55:42,038 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:42,038 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0a59441954132eb44621ade02029d4dc, daughterA=f91831a12657ef5da469c59d1a700126, daughterB=1f5b68527c02157d9f50f31363d383eb 2023-02-08 23:55:42,038 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:42,038 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 23:55:42,038 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:42,038 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:42,039 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,36713,1675900441074, table=testRollbackForSplitTableRegionWithReplica, region=7271d73f1b2300d37c9209b7cecf4ba1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=4a9ac7b252881a6ce15fd79f0bb87322, daughterA=d5d431db08f42353370bee8518763e27, daughterB=7ef76a86d8c8cb2ab967e57131b98db0 2023-02-08 23:55:42,039 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:42,039 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 23:55:42,039 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:42,039 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:42,039 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 23:55:42,039 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:42,040 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,040 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN 2023-02-08 23:55:42,040 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:42,040 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=add5f994245cd9a6365fc1ff87be30da, ASSIGN 2023-02-08 23:55:42,040 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a49dbb5e04b34d3140f2408d1509844, ASSIGN 2023-02-08 23:55:42,040 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 8a49dbb5e04b34d3140f2408d1509844, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:42,041 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure add5f994245cd9a6365fc1ff87be30da, server=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:42,041 DEBUG [Listener at localhost/37973] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 held the lock before restarting, call acquireLock to restore it. 2023-02-08 23:55:42,041 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,041 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,041 INFO [Listener at localhost/37973] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,041 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:42,041 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ca3eeec7d5e5bdf23a6f2e4cfc167e04, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:42,041 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=add5f994245cd9a6365fc1ff87be30da, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:42,041 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a49dbb5e04b34d3140f2408d1509844, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:42,041 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 8a49dbb5e04b34d3140f2408d1509844, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:42,041 DEBUG [Listener at localhost/37973] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure add5f994245cd9a6365fc1ff87be30da, server=jenkins-hbase4.apache.org,42343,1675900441151 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 23:55:42,041 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-08 23:55:42,042 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675900537183 2023-02-08 23:55:42,042 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 23:55:42,042 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675900537386 2023-02-08 23:55:42,042 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36199,1675900441120, table=hbase:meta, region=1588230740 2023-02-08 23:55:42,042 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675900537576 2023-02-08 23:55:42,043 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900572043 2023-02-08 23:55:42,044 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,044 DEBUG [Listener at localhost/37973] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:55:42,044 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675900537773 2023-02-08 23:55:42,045 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675900537941 2023-02-08 23:55:42,047 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675900538116 2023-02-08 23:55:42,047 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=6a82652058ff2f6139e2701c4c52385c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36199,1675900441120, regionLocation=jenkins-hbase4.apache.org,36199,1675900441120, openSeqNum=2 2023-02-08 23:55:42,048 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=ca3eeec7d5e5bdf23a6f2e4cfc167e04, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=2 2023-02-08 23:55:42,048 INFO [Listener at localhost/37973] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,42343,1675900441151, table=testRecoveryAndDoubleExecution, region=ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,048 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=add5f994245cd9a6365fc1ff87be30da, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=86 2023-02-08 23:55:42,048 DEBUG [Listener at localhost/37973] assignment.RegionStateStore(147): Load hbase:meta entry region=8a49dbb5e04b34d3140f2408d1509844, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42343,1675900441151, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151, openSeqNum=86 2023-02-08 23:55:42,048 INFO [Listener at localhost/37973] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 23:55:42,048 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900602048 2023-02-08 23:55:42,049 INFO [Listener at localhost/37973] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900662048 2023-02-08 23:55:42,049 INFO [Listener at localhost/37973] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 23:55:42,049 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 23:55:42,051 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ca3eeec7d5e5bdf23a6f2e4cfc167e04, daughterA=add5f994245cd9a6365fc1ff87be30da, daughterB=8a49dbb5e04b34d3140f2408d1509844 in 3.6750 sec 2023-02-08 23:55:42,057 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] throttle.PressureAwareThroughputController(145): 8a49dbb5e04b34d3140f2408d1509844#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:42,075 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/.tmp/cf2/cf49312340f74832b4e9b43bcab53be7 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/cf49312340f74832b4e9b43bcab53be7 2023-02-08 23:55:42,079 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8a49dbb5e04b34d3140f2408d1509844/cf2 of 8a49dbb5e04b34d3140f2408d1509844 into cf49312340f74832b4e9b43bcab53be7(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:42,079 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8a49dbb5e04b34d3140f2408d1509844: 2023-02-08 23:55:42,079 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844., storeName=8a49dbb5e04b34d3140f2408d1509844/cf2, priority=9, startTime=1675900542004; duration=0sec 2023-02-08 23:55:42,079 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:42,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42343] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:34096 deadline: 1675900602299, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04. is not online on jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:42,365 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/.tmp/cf1/603ac4ee6f7e4732a873820cb2c5e53d as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/603ac4ee6f7e4732a873820cb2c5e53d 2023-02-08 23:55:42,369 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in add5f994245cd9a6365fc1ff87be30da/cf1 of add5f994245cd9a6365fc1ff87be30da into 603ac4ee6f7e4732a873820cb2c5e53d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:42,369 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for add5f994245cd9a6365fc1ff87be30da: 2023-02-08 23:55:42,369 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da., storeName=add5f994245cd9a6365fc1ff87be30da/cf1, priority=9, startTime=1675900541917; duration=0sec 2023-02-08 23:55:42,369 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:42,415 DEBUG [Listener at localhost/37973] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 23:55:42,424 INFO [Listener at localhost/37973] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-08 23:55:42,426 DEBUG [Listener at localhost/37973] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 23:55:42,435 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:55:42,435 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:42,435 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:42,436 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:42,436 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:42,437 INFO [Listener at localhost/37973] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-08 23:55:42,437 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-08 23:55:42,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:42,440 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:42,440 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:42,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-08 23:55:42,441 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900542441"}]},"ts":"1675900542441"} 2023-02-08 23:55:42,442 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-08 23:55:42,444 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-08 23:55:42,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=add5f994245cd9a6365fc1ff87be30da, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a49dbb5e04b34d3140f2408d1509844, UNASSIGN}] 2023-02-08 23:55:42,445 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a49dbb5e04b34d3140f2408d1509844, UNASSIGN 2023-02-08 23:55:42,445 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=8a49dbb5e04b34d3140f2408d1509844, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:42,445 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900542445"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900542445"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900542445"}]},"ts":"1675900542445"} 2023-02-08 23:55:42,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 8a49dbb5e04b34d3140f2408d1509844, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:42,448 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=add5f994245cd9a6365fc1ff87be30da, UNASSIGN 2023-02-08 23:55:42,449 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=add5f994245cd9a6365fc1ff87be30da, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:42,449 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900542449"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900542449"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900542449"}]},"ts":"1675900542449"} 2023-02-08 23:55:42,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure add5f994245cd9a6365fc1ff87be30da, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:42,541 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:42,541 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:42,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-08 23:55:42,598 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close add5f994245cd9a6365fc1ff87be30da 2023-02-08 23:55:42,598 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing add5f994245cd9a6365fc1ff87be30da, disabling compactions & flushes 2023-02-08 23:55:42,598 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da. 2023-02-08 23:55:42,598 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da. 2023-02-08 23:55:42,598 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da. after waiting 0 ms 2023-02-08 23:55:42,598 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da. 2023-02-08 23:55:42,599 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a-bottom] to archive 2023-02-08 23:55:42,600 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:42,601 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,602 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,603 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,604 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,605 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,606 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,607 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,609 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58-bottom] to archive 2023-02-08 23:55:42,609 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:42,611 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,612 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,613 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,614 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,614 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,615 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,616 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 23:55:42,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:42,620 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da. 2023-02-08 23:55:42,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for add5f994245cd9a6365fc1ff87be30da: Waiting for close lock at 1675900542598Running coprocessor pre-close hooks at 1675900542598Disabling compacts and flushes for region at 1675900542598Disabling writes for close at 1675900542598Writing region close event to WAL at 1675900542617 (+19 ms)Running coprocessor post-close hooks at 1675900542620 (+3 ms)Closed at 1675900542620 2023-02-08 23:55:42,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed add5f994245cd9a6365fc1ff87be30da 2023-02-08 23:55:42,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8a49dbb5e04b34d3140f2408d1509844, disabling compactions & flushes 2023-02-08 23:55:42,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844. 2023-02-08 23:55:42,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844. 2023-02-08 23:55:42,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844. after waiting 0 ms 2023-02-08 23:55:42,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844. 2023-02-08 23:55:42,622 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=add5f994245cd9a6365fc1ff87be30da, regionState=CLOSED 2023-02-08 23:55:42,623 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900542622"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900542622"}]},"ts":"1675900542622"} 2023-02-08 23:55:42,623 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a-top] to archive 2023-02-08 23:55:42,624 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:42,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-08 23:55:42,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure add5f994245cd9a6365fc1ff87be30da, server=jenkins-hbase4.apache.org,42343,1675900441151 in 174 msec 2023-02-08 23:55:42,625 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,626 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=add5f994245cd9a6365fc1ff87be30da, UNASSIGN in 181 msec 2023-02-08 23:55:42,626 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,627 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,628 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,629 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,630 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,631 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,632 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58-top] to archive 2023-02-08 23:55:42,633 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:42,634 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,634 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,635 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,636 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,637 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,638 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,639 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,642 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 23:55:42,642 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:42,642 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844. 2023-02-08 23:55:42,642 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8a49dbb5e04b34d3140f2408d1509844: Waiting for close lock at 1675900542622Running coprocessor pre-close hooks at 1675900542622Disabling compacts and flushes for region at 1675900542622Disabling writes for close at 1675900542622Writing region close event to WAL at 1675900542640 (+18 ms)Running coprocessor post-close hooks at 1675900542642 (+2 ms)Closed at 1675900542642 2023-02-08 23:55:42,644 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,644 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=8a49dbb5e04b34d3140f2408d1509844, regionState=CLOSED 2023-02-08 23:55:42,644 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900542644"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900542644"}]},"ts":"1675900542644"} 2023-02-08 23:55:42,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-08 23:55:42,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 8a49dbb5e04b34d3140f2408d1509844, server=jenkins-hbase4.apache.org,42343,1675900441151 in 199 msec 2023-02-08 23:55:42,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-02-08 23:55:42,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a49dbb5e04b34d3140f2408d1509844, UNASSIGN in 203 msec 2023-02-08 23:55:42,649 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900542648"}]},"ts":"1675900542648"} 2023-02-08 23:55:42,649 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-08 23:55:42,652 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-08 23:55:42,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 215 msec 2023-02-08 23:55:42,742 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:42,743 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:42,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-08 23:55:42,743 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-08 23:55:42,743 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:42,744 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:42,744 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-08 23:55:42,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:42,746 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:42,746 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:42,746 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:42,747 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:42,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-08 23:55:42,751 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,751 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da 2023-02-08 23:55:42,751 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,753 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/recovered.edits] 2023-02-08 23:55:42,753 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/recovered.edits] 2023-02-08 23:55:42,753 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/recovered.edits] 2023-02-08 23:55:42,756 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/.regioninfo to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/.regioninfo 2023-02-08 23:55:42,757 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/.regioninfo to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/.regioninfo 2023-02-08 23:55:42,757 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/2fdb224c0a5c47e48f86524f230039fd 2023-02-08 23:55:42,758 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/3d9d315c61884930ae0f858b6483b912 2023-02-08 23:55:42,759 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,759 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/59ee7b1f36104eb88a48694ac5c8296d 2023-02-08 23:55:42,760 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/2fdb224c0a5c47e48f86524f230039fd.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,761 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,761 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/958f6123508c48df9585286422cd6935 2023-02-08 23:55:42,761 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/3d9d315c61884930ae0f858b6483b912.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,767 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/9b65b4f96314417fa8ae579ee2b0d748 2023-02-08 23:55:42,767 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,767 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/59ee7b1f36104eb88a48694ac5c8296d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,768 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/d3e5280d698c4567b2236f940aa4847a 2023-02-08 23:55:42,768 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,768 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/958f6123508c48df9585286422cd6935.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,769 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf1/e8d46dd70a3240ce84b42e36a5faf717 2023-02-08 23:55:42,769 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,769 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/9b65b4f96314417fa8ae579ee2b0d748.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,771 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,771 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/d3e5280d698c4567b2236f940aa4847a.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,772 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,772 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/08d96e436b5c4b8c9093b6092b4dde61 2023-02-08 23:55:42,772 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf1/e8d46dd70a3240ce84b42e36a5faf717.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,773 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/24820c13435040239cbdd81ac0da0dde 2023-02-08 23:55:42,775 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/56d5576c0b34420795371943899eb63d 2023-02-08 23:55:42,775 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,776 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/08d96e436b5c4b8c9093b6092b4dde61.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,776 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,776 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/94f78d66c9504dc9b0230e0a31733f58 2023-02-08 23:55:42,778 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/24820c13435040239cbdd81ac0da0dde.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,778 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/bf1e25331de544898c319348fee016a2 2023-02-08 23:55:42,778 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,779 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/56d5576c0b34420795371943899eb63d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,779 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/ed061e5440b64b8d919699c2db225c70 2023-02-08 23:55:42,779 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,780 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/94f78d66c9504dc9b0230e0a31733f58.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,780 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/cf2/f7ba7d1a9b2444c1b510d919547eff2d 2023-02-08 23:55:42,780 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,781 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/bf1e25331de544898c319348fee016a2.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,782 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,783 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/ed061e5440b64b8d919699c2db225c70.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,783 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/recovered.edits/85.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04/recovered.edits/85.seqid 2023-02-08 23:55:42,783 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/8a49dbb5e04b34d3140f2408d1509844/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,784 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,784 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/add5f994245cd9a6365fc1ff87be30da/cf2/f7ba7d1a9b2444c1b510d919547eff2d.ca3eeec7d5e5bdf23a6f2e4cfc167e04 2023-02-08 23:55:42,786 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/0986e7e15b3e40559aafc8d28fe075f0 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf1/0986e7e15b3e40559aafc8d28fe075f0 2023-02-08 23:55:42,786 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/603ac4ee6f7e4732a873820cb2c5e53d to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf1/603ac4ee6f7e4732a873820cb2c5e53d 2023-02-08 23:55:42,788 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/cf49312340f74832b4e9b43bcab53be7 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/cf2/cf49312340f74832b4e9b43bcab53be7 2023-02-08 23:55:42,789 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/58b4d818a02b4bf48a2017a22e9065bc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/cf2/58b4d818a02b4bf48a2017a22e9065bc 2023-02-08 23:55:42,791 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/recovered.edits/90.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844/recovered.edits/90.seqid 2023-02-08 23:55:42,792 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/recovered.edits/90.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da/recovered.edits/90.seqid 2023-02-08 23:55:42,792 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/8a49dbb5e04b34d3140f2408d1509844 2023-02-08 23:55:42,792 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testRecoveryAndDoubleExecution/add5f994245cd9a6365fc1ff87be30da 2023-02-08 23:55:42,792 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-08 23:55:42,794 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:42,796 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-08 23:55:42,798 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-08 23:55:42,798 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:42,798 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-08 23:55:42,798 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900542798"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:42,799 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900542798"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:42,799 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900542798"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:42,800 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 23:55:42,800 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ca3eeec7d5e5bdf23a6f2e4cfc167e04, NAME => 'testRecoveryAndDoubleExecution,,1675900536509.ca3eeec7d5e5bdf23a6f2e4cfc167e04.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => add5f994245cd9a6365fc1ff87be30da, NAME => 'testRecoveryAndDoubleExecution,,1675900538374.add5f994245cd9a6365fc1ff87be30da.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 8a49dbb5e04b34d3140f2408d1509844, NAME => 'testRecoveryAndDoubleExecution,41,1675900538374.8a49dbb5e04b34d3140f2408d1509844.', STARTKEY => '41', ENDKEY => ''}] 2023-02-08 23:55:42,800 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-08 23:55:42,800 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675900542800"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:42,801 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-08 23:55:42,803 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 23:55:42,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 59 msec 2023-02-08 23:55:42,847 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:42,847 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:42,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-08 23:55:42,848 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-08 23:55:42,863 INFO [Listener at localhost/37973] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=372 (was 364) - Thread LEAK? -, OpenFileDescriptor=828 (was 738) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=109 (was 94) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3060 (was 3096) 2023-02-08 23:55:42,877 INFO [Listener at localhost/37973] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=372, OpenFileDescriptor=828, MaxFileDescriptor=60000, SystemLoadAverage=109, ProcessCount=168, AvailableMemoryMB=3059 2023-02-08 23:55:42,877 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:55:42,877 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:42,877 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:42,879 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 23:55:42,879 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 23:55:42,880 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 23:55:42,881 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:55:42,882 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:55:42,883 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:42,884 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2 empty. 2023-02-08 23:55:42,884 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:42,884 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-08 23:55:42,892 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-08 23:55:42,893 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 5ff780cedf338f2c445fc440d550d8a2, NAME => 'testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp 2023-02-08 23:55:42,899 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:42,900 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 5ff780cedf338f2c445fc440d550d8a2, disabling compactions & flushes 2023-02-08 23:55:42,900 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:42,900 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:42,900 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. after waiting 0 ms 2023-02-08 23:55:42,900 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:42,900 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:42,900 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 5ff780cedf338f2c445fc440d550d8a2: Waiting for close lock at 1675900542900Disabling compacts and flushes for region at 1675900542900Disabling writes for close at 1675900542900Writing region close event to WAL at 1675900542900Closed at 1675900542900 2023-02-08 23:55:42,901 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:55:42,902 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900542902"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900542902"}]},"ts":"1675900542902"} 2023-02-08 23:55:42,903 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:55:42,904 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:55:42,904 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900542904"}]},"ts":"1675900542904"} 2023-02-08 23:55:42,905 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-08 23:55:42,908 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:55:42,909 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:55:42,909 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:55:42,909 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:55:42,909 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 23:55:42,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5ff780cedf338f2c445fc440d550d8a2, ASSIGN}] 2023-02-08 23:55:42,910 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5ff780cedf338f2c445fc440d550d8a2, ASSIGN 2023-02-08 23:55:42,911 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5ff780cedf338f2c445fc440d550d8a2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36713,1675900441074; forceNewPlan=false, retain=false 2023-02-08 23:55:43,061 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:55:43,062 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=5ff780cedf338f2c445fc440d550d8a2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,062 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900543062"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900543062"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900543062"}]},"ts":"1675900543062"} 2023-02-08 23:55:43,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 5ff780cedf338f2c445fc440d550d8a2, server=jenkins-hbase4.apache.org,36713,1675900441074}] 2023-02-08 23:55:43,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:43,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5ff780cedf338f2c445fc440d550d8a2, NAME => 'testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:55:43,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:55:43,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:43,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:43,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:43,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:43,222 INFO [StoreOpener-5ff780cedf338f2c445fc440d550d8a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:43,223 DEBUG [StoreOpener-5ff780cedf338f2c445fc440d550d8a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1 2023-02-08 23:55:43,223 DEBUG [StoreOpener-5ff780cedf338f2c445fc440d550d8a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1 2023-02-08 23:55:43,223 INFO [StoreOpener-5ff780cedf338f2c445fc440d550d8a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5ff780cedf338f2c445fc440d550d8a2 columnFamilyName cf1 2023-02-08 23:55:43,224 INFO [StoreOpener-5ff780cedf338f2c445fc440d550d8a2-1] regionserver.HStore(310): Store=5ff780cedf338f2c445fc440d550d8a2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:43,224 INFO [StoreOpener-5ff780cedf338f2c445fc440d550d8a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:43,225 DEBUG [StoreOpener-5ff780cedf338f2c445fc440d550d8a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2 2023-02-08 23:55:43,225 DEBUG [StoreOpener-5ff780cedf338f2c445fc440d550d8a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2 2023-02-08 23:55:43,225 INFO [StoreOpener-5ff780cedf338f2c445fc440d550d8a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5ff780cedf338f2c445fc440d550d8a2 columnFamilyName cf2 2023-02-08 23:55:43,225 INFO [StoreOpener-5ff780cedf338f2c445fc440d550d8a2-1] regionserver.HStore(310): Store=5ff780cedf338f2c445fc440d550d8a2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:43,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:43,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:43,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:43,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:43,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:55:43,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 5ff780cedf338f2c445fc440d550d8a2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60496983, jitterRate=-0.09852470457553864}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:43,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 5ff780cedf338f2c445fc440d550d8a2: Running coprocessor pre-open hook at 1675900543221Writing region info on filesystem at 1675900543221Initializing all the Stores at 1675900543221Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900543221Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900543221Cleaning up temporary data from old regions at 1675900543227 (+6 ms)Cleaning up detritus from prior splits at 1675900543227Running coprocessor post-open hooks at 1675900543230 (+3 ms)Region opened successfully at 1675900543230 2023-02-08 23:55:43,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2., pid=127, masterSystemTime=1675900543215 2023-02-08 23:55:43,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:43,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:43,233 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=5ff780cedf338f2c445fc440d550d8a2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,233 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900543233"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900543233"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900543233"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900543233"}]},"ts":"1675900543233"} 2023-02-08 23:55:43,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-08 23:55:43,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 5ff780cedf338f2c445fc440d550d8a2, server=jenkins-hbase4.apache.org,36713,1675900441074 in 170 msec 2023-02-08 23:55:43,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-08 23:55:43,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5ff780cedf338f2c445fc440d550d8a2, ASSIGN in 327 msec 2023-02-08 23:55:43,238 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:55:43,238 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900543238"}]},"ts":"1675900543238"} 2023-02-08 23:55:43,239 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-08 23:55:43,240 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:55:43,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 361 msec 2023-02-08 23:55:43,383 DEBUG [Listener at localhost/37973] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:55:43,386 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43382, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:55:43,391 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:43,391 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:43,392 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:43,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,393 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:43,393 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,393 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 23:55:43,393 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 23:55:43,393 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,393 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,395 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,395 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,395 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,395 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,395 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,395 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,396 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,396 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,396 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:43,396 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,397 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:43,397 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:43,397 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:43,397 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,396 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:43,397 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:43,397 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:43,397 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:43,397 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:43,397 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:43,397 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:43,397 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:43,397 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. started... 2023-02-08 23:55:43,398 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5ff780cedf338f2c445fc440d550d8a2 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 23:55:43,399 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,399 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,399 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,399 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,399 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:43,399 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:43,406 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/787b4e1a353d479484868379d912727c 2023-02-08 23:55:43,417 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/e5c43717a62e4fd1887e4599baf22d99 2023-02-08 23:55:43,421 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/787b4e1a353d479484868379d912727c as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/787b4e1a353d479484868379d912727c 2023-02-08 23:55:43,424 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/787b4e1a353d479484868379d912727c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:55:43,425 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/e5c43717a62e4fd1887e4599baf22d99 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/e5c43717a62e4fd1887e4599baf22d99 2023-02-08 23:55:43,427 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/e5c43717a62e4fd1887e4599baf22d99, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:55:43,428 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 5ff780cedf338f2c445fc440d550d8a2 in 30ms, sequenceid=6, compaction requested=false 2023-02-08 23:55:43,428 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:43,429 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5ff780cedf338f2c445fc440d550d8a2: 2023-02-08 23:55:43,429 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:43,429 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:43,429 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:43,429 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,429 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:43,429 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:43,431 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,431 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:43,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:43,432 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,432 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:43,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:43,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:43,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:43,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 23:55:43,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3840df6a[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:43,433 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 23:55:43,433 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,434 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,434 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,434 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:43,434 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:43,435 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:43,434 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,435 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:43,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,434 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:43,435 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:43,435 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:43,435 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:43,434 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,435 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:43,435 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:43,436 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:43,436 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,436 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:43,436 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:43,436 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,436 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,436 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:43,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:43,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:43,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:43,438 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:43,438 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,438 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:43,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:43,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:43,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36713,1675900441074': 2023-02-08 23:55:43,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:55:43,440 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 23:55:43,440 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:43,440 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:43,440 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,440 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:43,442 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,442 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,442 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,442 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,442 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,442 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:43,442 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:43,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,442 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,442 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,443 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:43,444 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,445 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,445 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,450 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,450 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,450 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,450 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,450 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,450 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,450 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,450 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:43,451 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:43,450 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,450 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,450 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,451 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:43,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,451 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,451 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:43,451 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,451 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,451 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,451 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,451 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,451 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,451 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,551 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:43,552 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:43,552 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:43,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:43,562 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:43,562 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:43,562 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:43,564 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:43,564 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,564 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:43,564 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,564 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 23:55:43,564 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 23:55:43,564 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,564 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,566 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,566 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,566 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,566 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,566 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,566 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,567 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,569 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:43,569 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,569 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,569 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:43,569 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,569 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:43,570 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:43,570 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:43,570 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:43,570 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. started... 2023-02-08 23:55:43,570 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5ff780cedf338f2c445fc440d550d8a2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:43,570 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,572 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,572 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:43,572 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,572 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,573 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:43,582 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/3b2a7b13dd1b489ca09de59cc423fece 2023-02-08 23:55:43,591 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/4e3091e07c2b4629ae6f968ffb59fee2 2023-02-08 23:55:43,595 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/3b2a7b13dd1b489ca09de59cc423fece as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/3b2a7b13dd1b489ca09de59cc423fece 2023-02-08 23:55:43,598 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/3b2a7b13dd1b489ca09de59cc423fece, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:55:43,599 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/4e3091e07c2b4629ae6f968ffb59fee2 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e3091e07c2b4629ae6f968ffb59fee2 2023-02-08 23:55:43,601 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e3091e07c2b4629ae6f968ffb59fee2, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:55:43,602 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5ff780cedf338f2c445fc440d550d8a2 in 32ms, sequenceid=19, compaction requested=false 2023-02-08 23:55:43,602 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5ff780cedf338f2c445fc440d550d8a2: 2023-02-08 23:55:43,602 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:43,602 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:43,602 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:43,602 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,602 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:43,602 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:43,604 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,604 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:43,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:43,604 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,604 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:43,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:43,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:43,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:43,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 23:55:43,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@357611aa[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:43,606 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 23:55:43,606 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,607 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,607 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,607 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,607 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,607 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:43,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,607 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:43,608 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:43,608 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:43,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,608 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:43,608 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:43,608 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:43,608 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:43,608 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:43,608 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:43,609 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:43,609 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,609 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:43,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:43,609 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:43,609 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:43,609 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,609 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:43,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:43,611 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:43,611 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,611 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:43,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:43,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:43,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36713,1675900441074': 2023-02-08 23:55:43,613 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:55:43,613 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 23:55:43,613 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:43,613 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:43,613 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,613 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:43,615 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,615 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:43,615 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,615 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,615 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,615 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,615 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,615 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,616 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,616 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:43,618 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,618 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,618 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,620 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,620 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,620 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,620 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,620 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,620 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,621 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,620 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:43,621 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:43,620 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,620 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,621 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,621 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,621 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:43,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,621 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:43,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,621 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,621 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,622 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,622 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,622 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,622 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,622 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,721 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:43,722 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:43,722 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:43,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:43,731 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:43,732 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:43,732 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:43,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:43,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,733 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:43,733 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,734 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 23:55:43,734 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 23:55:43,734 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,734 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,735 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,735 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,735 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,735 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,735 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,735 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:43,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,736 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,737 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:43,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,737 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:43,737 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:43,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,737 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. started... 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:43,737 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5ff780cedf338f2c445fc440d550d8a2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:43,739 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,740 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,740 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,740 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:43,740 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,740 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:43,745 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/44c787fdaba3472382b81588f062dcb5 2023-02-08 23:55:43,753 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/4e099f82c64f4d3fa4ec030b486b560a 2023-02-08 23:55:43,757 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/44c787fdaba3472382b81588f062dcb5 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/44c787fdaba3472382b81588f062dcb5 2023-02-08 23:55:43,760 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/44c787fdaba3472382b81588f062dcb5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:55:43,760 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/4e099f82c64f4d3fa4ec030b486b560a as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e099f82c64f4d3fa4ec030b486b560a 2023-02-08 23:55:43,763 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e099f82c64f4d3fa4ec030b486b560a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:55:43,764 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5ff780cedf338f2c445fc440d550d8a2 in 27ms, sequenceid=32, compaction requested=true 2023-02-08 23:55:43,764 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5ff780cedf338f2c445fc440d550d8a2: 2023-02-08 23:55:43,764 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:43,764 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:43,764 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:43,764 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,764 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:43,764 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:43,765 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,765 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:43,766 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,766 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:43,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:43,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:43,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:43,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 23:55:43,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7589765e[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:43,767 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 23:55:43,767 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,768 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,769 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:43,768 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,769 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,769 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:43,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:43,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:43,774 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:43,774 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,774 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,774 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:43,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:43,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:43,775 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:43,775 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,775 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36713,1675900441074': 2023-02-08 23:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:55:43,777 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 23:55:43,777 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:43,777 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:43,777 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,777 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:43,779 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,779 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,780 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,779 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,779 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,780 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,780 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:43,780 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,780 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,781 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:43,782 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,783 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,783 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,785 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:43,785 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:43,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,786 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:43,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,785 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,785 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,786 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,786 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,786 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:43,786 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,786 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,786 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,787 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,787 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,787 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,787 ERROR [BP-379259138-172.31.14.131-1675900436765 heartbeating to localhost/127.0.0.1:38079] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-379259138-172.31.14.131-1675900436765 (Datanode Uuid c96d9a8d-7822-404e-9335-ec30f9bdbf87) service to localhost/127.0.0.1:38079 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 23:55:43,886 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:43,886 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:43,887 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:43,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:43,897 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:43,897 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:43,898 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:43,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:43,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,899 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:43,899 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,899 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 23:55:43,899 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 23:55:43,899 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,899 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,900 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,900 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,900 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,900 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,901 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,901 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:43,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,901 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,901 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,902 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:43,902 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,902 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,902 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:43,903 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:43,903 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:43,903 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:43,903 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:43,903 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:43,903 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:43,903 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:43,903 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. started... 2023-02-08 23:55:43,903 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5ff780cedf338f2c445fc440d550d8a2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:43,904 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,906 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,906 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,906 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:43,906 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,906 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:43,911 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/10153ec5089d46a7bb2b66cb21fdc5bb 2023-02-08 23:55:43,926 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/a237c358ab424b3eb42d514292a522b2 2023-02-08 23:55:43,930 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/10153ec5089d46a7bb2b66cb21fdc5bb as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/10153ec5089d46a7bb2b66cb21fdc5bb 2023-02-08 23:55:43,933 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/10153ec5089d46a7bb2b66cb21fdc5bb, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:55:43,933 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/a237c358ab424b3eb42d514292a522b2 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/a237c358ab424b3eb42d514292a522b2 2023-02-08 23:55:43,936 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/a237c358ab424b3eb42d514292a522b2, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:55:43,937 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5ff780cedf338f2c445fc440d550d8a2 in 34ms, sequenceid=45, compaction requested=true 2023-02-08 23:55:43,937 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5ff780cedf338f2c445fc440d550d8a2: 2023-02-08 23:55:43,938 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:43,939 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,939 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:43,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:43,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:43,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:43,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:43,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 23:55:43,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31bd18dd[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:43,942 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 23:55:43,942 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,943 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,943 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,943 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,943 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,943 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:43,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:43,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:43,944 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:43,943 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:43,944 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:43,944 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:43,944 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:43,945 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,945 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:43,947 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:43,947 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,947 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:43,947 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:43,947 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,947 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:43,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:43,947 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36713,1675900441074': 2023-02-08 23:55:43,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:55:43,950 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 23:55:43,950 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:43,950 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:43,950 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,950 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:43,951 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,951 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,951 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,951 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:43,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:43,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,951 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,951 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,951 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:43,952 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,954 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:43,955 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:43,956 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,956 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,956 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,961 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,961 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,961 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,961 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,961 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:43,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,962 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:43,961 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:43,961 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,961 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,961 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:43,962 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,962 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,961 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,962 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:43,962 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:43,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,962 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,962 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:43,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,962 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:43,962 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:43,963 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:43,963 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:43,963 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,063 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:44,063 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:44,063 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:44,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:44,075 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:44,076 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:44,076 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:44,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:44,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,078 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:44,078 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:44,078 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 23:55:44,078 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 23:55:44,078 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,078 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,080 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,080 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,080 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,084 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,084 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,084 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,086 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:44,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:44,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:44,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:44,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:44,086 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:44,086 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. started... 2023-02-08 23:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:44,087 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:44,087 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5ff780cedf338f2c445fc440d550d8a2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:44,088 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,088 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,088 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:44,092 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,092 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,092 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:44,118 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/2576c52ed53c41548ab0552080039fce 2023-02-08 23:55:44,130 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/c14da944b6fa4d8f894bf0da679a9cb0 2023-02-08 23:55:44,133 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/2576c52ed53c41548ab0552080039fce as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/2576c52ed53c41548ab0552080039fce 2023-02-08 23:55:44,136 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/2576c52ed53c41548ab0552080039fce, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:55:44,137 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/c14da944b6fa4d8f894bf0da679a9cb0 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/c14da944b6fa4d8f894bf0da679a9cb0 2023-02-08 23:55:44,140 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/c14da944b6fa4d8f894bf0da679a9cb0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:55:44,141 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5ff780cedf338f2c445fc440d550d8a2 in 54ms, sequenceid=58, compaction requested=true 2023-02-08 23:55:44,141 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5ff780cedf338f2c445fc440d550d8a2: 2023-02-08 23:55:44,141 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:44,141 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:44,141 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:44,141 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,141 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:44,142 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:44,144 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,144 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:44,144 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,144 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:44,146 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 23:55:44,146 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fb83a0f[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:44,146 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 23:55:44,146 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,147 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,147 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,148 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:44,148 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,148 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:44,147 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,148 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:44,148 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:44,148 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:44,148 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:44,148 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:44,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,148 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:44,148 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:44,148 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:44,149 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:44,149 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:44,149 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:44,149 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:44,149 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:44,149 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:44,150 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:44,150 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,150 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:44,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:44,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:44,150 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:44,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:44,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:44,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:44,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36713,1675900441074': 2023-02-08 23:55:44,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:55:44,153 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 23:55:44,153 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:44,153 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:44,153 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,153 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:44,161 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,161 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,161 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,161 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,161 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:44,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:44,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,161 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,161 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,161 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:44,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,162 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,162 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:44,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,163 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,164 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:44,164 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,166 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,166 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,166 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,166 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,166 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,166 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,166 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,166 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:44,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,166 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,166 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,167 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,167 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,166 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:44,167 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:44,166 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,167 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,167 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,167 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,167 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:44,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,167 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:44,167 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,167 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,267 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:44,268 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:44,268 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:44,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:44,277 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:44,277 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:44,278 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:44,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:44,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,280 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:44,280 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:44,280 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 23:55:44,280 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 23:55:44,280 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,280 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,281 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,281 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,281 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,281 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,282 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,282 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:44,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,282 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,283 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,283 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:44,283 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,283 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:44,283 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:44,283 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 23:55:44,283 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:44,283 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:44,283 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:44,283 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,283 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 23:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:44,284 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:44,284 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. started... 2023-02-08 23:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 23:55:44,284 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5ff780cedf338f2c445fc440d550d8a2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:44,285 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,288 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,288 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,288 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:44,288 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,288 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:44,295 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/1f4d5e17c0ba4488a52526f12c647534 2023-02-08 23:55:44,306 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/f830532c24074658900c91541721fce8 2023-02-08 23:55:44,310 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/1f4d5e17c0ba4488a52526f12c647534 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/1f4d5e17c0ba4488a52526f12c647534 2023-02-08 23:55:44,313 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/1f4d5e17c0ba4488a52526f12c647534, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:55:44,314 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/f830532c24074658900c91541721fce8 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/f830532c24074658900c91541721fce8 2023-02-08 23:55:44,317 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/f830532c24074658900c91541721fce8, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:55:44,318 INFO [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5ff780cedf338f2c445fc440d550d8a2 in 34ms, sequenceid=71, compaction requested=true 2023-02-08 23:55:44,318 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5ff780cedf338f2c445fc440d550d8a2: 2023-02-08 23:55:44,318 DEBUG [rs(jenkins-hbase4.apache.org,36713,1675900441074)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:44,318 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:44,318 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:44,318 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,318 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 23:55:44,318 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 23:55:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,319 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:44,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,320 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:44,321 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 23:55:44,321 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ac68113[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:44,321 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 23:55:44,321 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,326 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,326 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,326 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:44,327 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:44,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,327 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:44,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 23:55:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 23:55:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:44,329 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:44,329 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:44,329 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:44,329 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:44,329 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:44,330 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:44,330 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:44,330 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 23:55:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:44,330 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:44,330 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 23:55:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:44,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36713,1675900441074': 2023-02-08 23:55:44,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 23:55:44,333 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 23:55:44,333 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:44,333 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:44,333 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,333 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:44,334 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,334 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,334 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,334 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,334 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:44,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:44,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,334 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,334 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,334 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,335 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,335 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:44,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,337 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:44,337 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,337 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,341 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 23:55:44,341 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:44,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,341 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,341 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:44,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,341 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:44,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,341 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,341 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,341 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:44,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,341 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,342 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,342 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:44,342 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,342 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,342 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:44,342 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,342 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 23:55:44,441 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:44,442 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:44,442 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:44,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:44,448 DEBUG [Listener at localhost/37973] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,449 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=5ff780cedf338f2c445fc440d550d8a2, daughterA=2d53e24f43c86247fa3362b893725806, daughterB=338ee863654205b6854621d67961edb9 2023-02-08 23:55:44,450 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=5ff780cedf338f2c445fc440d550d8a2, daughterA=2d53e24f43c86247fa3362b893725806, daughterB=338ee863654205b6854621d67961edb9 2023-02-08 23:55:44,450 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=5ff780cedf338f2c445fc440d550d8a2, daughterA=2d53e24f43c86247fa3362b893725806, daughterB=338ee863654205b6854621d67961edb9 2023-02-08 23:55:44,450 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=5ff780cedf338f2c445fc440d550d8a2, daughterA=2d53e24f43c86247fa3362b893725806, daughterB=338ee863654205b6854621d67961edb9 2023-02-08 23:55:44,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5ff780cedf338f2c445fc440d550d8a2, UNASSIGN}] 2023-02-08 23:55:44,452 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5ff780cedf338f2c445fc440d550d8a2, UNASSIGN 2023-02-08 23:55:44,453 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=5ff780cedf338f2c445fc440d550d8a2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,453 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900544453"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900544453"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900544453"}]},"ts":"1675900544453"} 2023-02-08 23:55:44,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 5ff780cedf338f2c445fc440d550d8a2, server=jenkins-hbase4.apache.org,36713,1675900441074}] 2023-02-08 23:55:44,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 5ff780cedf338f2c445fc440d550d8a2, disabling compactions & flushes 2023-02-08 23:55:44,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:44,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:44,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. after waiting 0 ms 2023-02-08 23:55:44,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:44,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 5ff780cedf338f2c445fc440d550d8a2 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 23:55:44,615 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/449179b3fd5149809b3583f312303ec3 2023-02-08 23:55:44,626 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/dbcf14fcf1d94353a8f2aa3f47cd1045 2023-02-08 23:55:44,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf1/449179b3fd5149809b3583f312303ec3 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/449179b3fd5149809b3583f312303ec3 2023-02-08 23:55:44,633 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/449179b3fd5149809b3583f312303ec3, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:55:44,634 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.tmp/cf2/dbcf14fcf1d94353a8f2aa3f47cd1045 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/dbcf14fcf1d94353a8f2aa3f47cd1045 2023-02-08 23:55:44,639 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/dbcf14fcf1d94353a8f2aa3f47cd1045, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:55:44,640 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 5ff780cedf338f2c445fc440d550d8a2 in 35ms, sequenceid=82, compaction requested=true 2023-02-08 23:55:44,650 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 23:55:44,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:44,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. 2023-02-08 23:55:44,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 5ff780cedf338f2c445fc440d550d8a2: Waiting for close lock at 1675900544605Running coprocessor pre-close hooks at 1675900544605Disabling compacts and flushes for region at 1675900544605Disabling writes for close at 1675900544605Obtaining lock to block concurrent updates at 1675900544605Preparing flush snapshotting stores in 5ff780cedf338f2c445fc440d550d8a2 at 1675900544605Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675900544606 (+1 ms)Flushing stores of testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. at 1675900544606Flushing 5ff780cedf338f2c445fc440d550d8a2/cf1: creating writer at 1675900544608 (+2 ms)Flushing 5ff780cedf338f2c445fc440d550d8a2/cf1: appending metadata at 1675900544610 (+2 ms)Flushing 5ff780cedf338f2c445fc440d550d8a2/cf1: closing flushed file at 1675900544610Flushing 5ff780cedf338f2c445fc440d550d8a2/cf2: creating writer at 1675900544619 (+9 ms)Flushing 5ff780cedf338f2c445fc440d550d8a2/cf2: appending metadata at 1675900544621 (+2 ms)Flushing 5ff780cedf338f2c445fc440d550d8a2/cf2: closing flushed file at 1675900544621Flushing 5ff780cedf338f2c445fc440d550d8a2/cf1: reopening flushed file at 1675900544630 (+9 ms)Flushing 5ff780cedf338f2c445fc440d550d8a2/cf2: reopening flushed file at 1675900544635 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 5ff780cedf338f2c445fc440d550d8a2 in 35ms, sequenceid=82, compaction requested=true at 1675900544640 (+5 ms)Writing region close event to WAL at 1675900544645 (+5 ms)Running coprocessor post-close hooks at 1675900544651 (+6 ms)Closed at 1675900544651 2023-02-08 23:55:44,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,653 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=5ff780cedf338f2c445fc440d550d8a2, regionState=CLOSED 2023-02-08 23:55:44,653 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900544653"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900544653"}]},"ts":"1675900544653"} 2023-02-08 23:55:44,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-08 23:55:44,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 5ff780cedf338f2c445fc440d550d8a2, server=jenkins-hbase4.apache.org,36713,1675900441074 in 201 msec 2023-02-08 23:55:44,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-08 23:55:44,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5ff780cedf338f2c445fc440d550d8a2, UNASSIGN in 204 msec 2023-02-08 23:55:44,672 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=5ff780cedf338f2c445fc440d550d8a2, threads=14 2023-02-08 23:55:44,673 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e099f82c64f4d3fa4ec030b486b560a for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,673 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e3091e07c2b4629ae6f968ffb59fee2 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,673 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/a237c358ab424b3eb42d514292a522b2 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,673 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/c14da944b6fa4d8f894bf0da679a9cb0 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,673 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/dbcf14fcf1d94353a8f2aa3f47cd1045 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,673 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/e5c43717a62e4fd1887e4599baf22d99 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,674 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/10153ec5089d46a7bb2b66cb21fdc5bb for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,673 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/f830532c24074658900c91541721fce8 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,674 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/1f4d5e17c0ba4488a52526f12c647534 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,674 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/2576c52ed53c41548ab0552080039fce for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,674 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/3b2a7b13dd1b489ca09de59cc423fece for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,674 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/449179b3fd5149809b3583f312303ec3 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,674 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/44c787fdaba3472382b81588f062dcb5 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,675 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/787b4e1a353d479484868379d912727c for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,681 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e099f82c64f4d3fa4ec030b486b560a, top=false 2023-02-08 23:55:44,682 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/2576c52ed53c41548ab0552080039fce, top=false 2023-02-08 23:55:44,682 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e3091e07c2b4629ae6f968ffb59fee2, top=false 2023-02-08 23:55:44,685 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/1f4d5e17c0ba4488a52526f12c647534, top=false 2023-02-08 23:55:44,685 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/a237c358ab424b3eb42d514292a522b2, top=false 2023-02-08 23:55:44,688 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/449179b3fd5149809b3583f312303ec3, top=false 2023-02-08 23:55:44,688 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/dbcf14fcf1d94353a8f2aa3f47cd1045, top=false 2023-02-08 23:55:44,690 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/787b4e1a353d479484868379d912727c, top=false 2023-02-08 23:55:44,692 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/10153ec5089d46a7bb2b66cb21fdc5bb, top=false 2023-02-08 23:55:44,693 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/f830532c24074658900c91541721fce8, top=false 2023-02-08 23:55:44,693 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/3b2a7b13dd1b489ca09de59cc423fece, top=false 2023-02-08 23:55:44,697 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/e5c43717a62e4fd1887e4599baf22d99, top=false 2023-02-08 23:55:44,698 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/44c787fdaba3472382b81588f062dcb5, top=false 2023-02-08 23:55:44,699 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/c14da944b6fa4d8f894bf0da679a9cb0, top=false 2023-02-08 23:55:44,707 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-f830532c24074658900c91541721fce8 for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,707 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e099f82c64f4d3fa4ec030b486b560a for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,707 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-3b2a7b13dd1b489ca09de59cc423fece for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,708 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-dbcf14fcf1d94353a8f2aa3f47cd1045 for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,711 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-44c787fdaba3472382b81588f062dcb5 for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,714 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e099f82c64f4d3fa4ec030b486b560a for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,716 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/dbcf14fcf1d94353a8f2aa3f47cd1045 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,716 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/f830532c24074658900c91541721fce8 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,717 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/3b2a7b13dd1b489ca09de59cc423fece for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,718 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/44c787fdaba3472382b81588f062dcb5 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,720 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-e5c43717a62e4fd1887e4599baf22d99 for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,720 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-1f4d5e17c0ba4488a52526f12c647534 for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,720 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-a237c358ab424b3eb42d514292a522b2 for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,721 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e3091e07c2b4629ae6f968ffb59fee2 for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,721 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-449179b3fd5149809b3583f312303ec3 for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,721 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-10153ec5089d46a7bb2b66cb21fdc5bb for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,721 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-2576c52ed53c41548ab0552080039fce for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,722 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-c14da944b6fa4d8f894bf0da679a9cb0 for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,722 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/.splits/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-787b4e1a353d479484868379d912727c for child: 2d53e24f43c86247fa3362b893725806, parent: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,725 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/a237c358ab424b3eb42d514292a522b2 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,725 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/1f4d5e17c0ba4488a52526f12c647534 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,725 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/449179b3fd5149809b3583f312303ec3 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,726 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/e5c43717a62e4fd1887e4599baf22d99 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,727 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/10153ec5089d46a7bb2b66cb21fdc5bb for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,727 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e3091e07c2b4629ae6f968ffb59fee2 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,728 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/2576c52ed53c41548ab0552080039fce for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,728 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/787b4e1a353d479484868379d912727c for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,728 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/c14da944b6fa4d8f894bf0da679a9cb0 for region: 5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:44,728 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 5ff780cedf338f2c445fc440d550d8a2 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-08 23:55:44,752 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 23:55:44,753 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 23:55:44,754 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675900544754"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1675900544754"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1675900544754"}]},"ts":"1675900544754"} 2023-02-08 23:55:44,755 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900544754"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900544754"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900544754"}]},"ts":"1675900544754"} 2023-02-08 23:55:44,755 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900544754"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900544754"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900544754"}]},"ts":"1675900544754"} 2023-02-08 23:55:44,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2d53e24f43c86247fa3362b893725806, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=338ee863654205b6854621d67961edb9, ASSIGN}] 2023-02-08 23:55:44,758 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=338ee863654205b6854621d67961edb9, ASSIGN 2023-02-08 23:55:44,758 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=338ee863654205b6854621d67961edb9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36713,1675900441074; forceNewPlan=false, retain=false 2023-02-08 23:55:44,759 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2d53e24f43c86247fa3362b893725806, ASSIGN 2023-02-08 23:55:44,759 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2d53e24f43c86247fa3362b893725806, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36713,1675900441074; forceNewPlan=false, retain=false 2023-02-08 23:55:44,865 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:55:44,909 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 23:55:44,910 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=338ee863654205b6854621d67961edb9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,910 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900544910"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900544910"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900544910"}]},"ts":"1675900544910"} 2023-02-08 23:55:44,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 338ee863654205b6854621d67961edb9, server=jenkins-hbase4.apache.org,36713,1675900441074}] 2023-02-08 23:55:44,914 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=2d53e24f43c86247fa3362b893725806, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:44,914 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900544914"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900544914"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900544914"}]},"ts":"1675900544914"} 2023-02-08 23:55:44,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 2d53e24f43c86247fa3362b893725806, server=jenkins-hbase4.apache.org,36713,1675900441074}] 2023-02-08 23:55:45,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9. 2023-02-08 23:55:45,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 338ee863654205b6854621d67961edb9, NAME => 'testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9.', STARTKEY => '71', ENDKEY => ''} 2023-02-08 23:55:45,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:55:45,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 338ee863654205b6854621d67961edb9 2023-02-08 23:55:45,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:45,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 338ee863654205b6854621d67961edb9 2023-02-08 23:55:45,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 338ee863654205b6854621d67961edb9 2023-02-08 23:55:45,067 INFO [StoreOpener-338ee863654205b6854621d67961edb9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 338ee863654205b6854621d67961edb9 2023-02-08 23:55:45,069 DEBUG [StoreOpener-338ee863654205b6854621d67961edb9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9/cf1 2023-02-08 23:55:45,069 DEBUG [StoreOpener-338ee863654205b6854621d67961edb9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9/cf1 2023-02-08 23:55:45,069 INFO [StoreOpener-338ee863654205b6854621d67961edb9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 338ee863654205b6854621d67961edb9 columnFamilyName cf1 2023-02-08 23:55:45,070 INFO [StoreOpener-338ee863654205b6854621d67961edb9-1] regionserver.HStore(310): Store=338ee863654205b6854621d67961edb9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:45,070 INFO [StoreOpener-338ee863654205b6854621d67961edb9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 338ee863654205b6854621d67961edb9 2023-02-08 23:55:45,071 DEBUG [StoreOpener-338ee863654205b6854621d67961edb9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9/cf2 2023-02-08 23:55:45,071 DEBUG [StoreOpener-338ee863654205b6854621d67961edb9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9/cf2 2023-02-08 23:55:45,071 INFO [StoreOpener-338ee863654205b6854621d67961edb9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 338ee863654205b6854621d67961edb9 columnFamilyName cf2 2023-02-08 23:55:45,072 INFO [StoreOpener-338ee863654205b6854621d67961edb9-1] regionserver.HStore(310): Store=338ee863654205b6854621d67961edb9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:45,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9 2023-02-08 23:55:45,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9 2023-02-08 23:55:45,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:45,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 338ee863654205b6854621d67961edb9 2023-02-08 23:55:45,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 338ee863654205b6854621d67961edb9; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61486154, jitterRate=-0.08378490805625916}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:45,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 338ee863654205b6854621d67961edb9: Running coprocessor pre-open hook at 1675900545067Writing region info on filesystem at 1675900545067Initializing all the Stores at 1675900545067Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900545067Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900545067Cleaning up temporary data from old regions at 1675900545074 (+7 ms)Cleaning up detritus from prior splits at 1675900545074Running coprocessor post-open hooks at 1675900545076 (+2 ms)Region opened successfully at 1675900545076 2023-02-08 23:55:45,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9., pid=133, masterSystemTime=1675900545063 2023-02-08 23:55:45,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9. 2023-02-08 23:55:45,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9. 2023-02-08 23:55:45,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806. 2023-02-08 23:55:45,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2d53e24f43c86247fa3362b893725806, NAME => 'testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.', STARTKEY => '', ENDKEY => '71'} 2023-02-08 23:55:45,078 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=338ee863654205b6854621d67961edb9, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:45,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:55:45,078 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900545078"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900545078"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900545078"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900545078"}]},"ts":"1675900545078"} 2023-02-08 23:55:45,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 2d53e24f43c86247fa3362b893725806 2023-02-08 23:55:45,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:45,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2d53e24f43c86247fa3362b893725806 2023-02-08 23:55:45,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2d53e24f43c86247fa3362b893725806 2023-02-08 23:55:45,080 INFO [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2d53e24f43c86247fa3362b893725806 2023-02-08 23:55:45,081 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1 2023-02-08 23:55:45,081 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1 2023-02-08 23:55:45,081 INFO [StoreOpener-2d53e24f43c86247fa3362b893725806-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d53e24f43c86247fa3362b893725806 columnFamilyName cf1 2023-02-08 23:55:45,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-08 23:55:45,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 338ee863654205b6854621d67961edb9, server=jenkins-hbase4.apache.org,36713,1675900441074 in 170 msec 2023-02-08 23:55:45,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=338ee863654205b6854621d67961edb9, ASSIGN in 327 msec 2023-02-08 23:55:45,090 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-10153ec5089d46a7bb2b66cb21fdc5bb 2023-02-08 23:55:45,092 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-1f4d5e17c0ba4488a52526f12c647534 2023-02-08 23:55:45,095 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-2576c52ed53c41548ab0552080039fce 2023-02-08 23:55:45,098 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-3b2a7b13dd1b489ca09de59cc423fece 2023-02-08 23:55:45,100 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-449179b3fd5149809b3583f312303ec3 2023-02-08 23:55:45,103 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-44c787fdaba3472382b81588f062dcb5 2023-02-08 23:55:45,105 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-787b4e1a353d479484868379d912727c 2023-02-08 23:55:45,106 INFO [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(310): Store=2d53e24f43c86247fa3362b893725806/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:45,106 INFO [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2d53e24f43c86247fa3362b893725806 2023-02-08 23:55:45,106 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2 2023-02-08 23:55:45,107 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2 2023-02-08 23:55:45,107 INFO [StoreOpener-2d53e24f43c86247fa3362b893725806-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d53e24f43c86247fa3362b893725806 columnFamilyName cf2 2023-02-08 23:55:45,111 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e099f82c64f4d3fa4ec030b486b560a 2023-02-08 23:55:45,113 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e3091e07c2b4629ae6f968ffb59fee2 2023-02-08 23:55:45,115 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-a237c358ab424b3eb42d514292a522b2 2023-02-08 23:55:45,118 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-c14da944b6fa4d8f894bf0da679a9cb0 2023-02-08 23:55:45,120 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-dbcf14fcf1d94353a8f2aa3f47cd1045 2023-02-08 23:55:45,122 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-e5c43717a62e4fd1887e4599baf22d99 2023-02-08 23:55:45,125 DEBUG [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-f830532c24074658900c91541721fce8 2023-02-08 23:55:45,125 INFO [StoreOpener-2d53e24f43c86247fa3362b893725806-1] regionserver.HStore(310): Store=2d53e24f43c86247fa3362b893725806/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:45,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806 2023-02-08 23:55:45,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806 2023-02-08 23:55:45,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:45,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2d53e24f43c86247fa3362b893725806 2023-02-08 23:55:45,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2d53e24f43c86247fa3362b893725806; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68778664, jitterRate=0.024881958961486816}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:45,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2d53e24f43c86247fa3362b893725806: Running coprocessor pre-open hook at 1675900545079Writing region info on filesystem at 1675900545079Initializing all the Stores at 1675900545079Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900545079Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900545080 (+1 ms)Cleaning up temporary data from old regions at 1675900545126 (+46 ms)Cleaning up detritus from prior splits at 1675900545127 (+1 ms)Running coprocessor post-open hooks at 1675900545129 (+2 ms)Region opened successfully at 1675900545129 2023-02-08 23:55:45,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806., pid=134, masterSystemTime=1675900545063 2023-02-08 23:55:45,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:45,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:55:45,131 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:55:45,131 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:55:45,131 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-08 23:55:45,131 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-08 23:55:45,131 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] regionserver.HStore(1912): 2d53e24f43c86247fa3362b893725806/cf2 is initiating minor compaction (all files) 2023-02-08 23:55:45,131 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] regionserver.HStore(1912): 2d53e24f43c86247fa3362b893725806/cf1 is initiating minor compaction (all files) 2023-02-08 23:55:45,131 INFO [RS:0;jenkins-hbase4:36713-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 2d53e24f43c86247fa3362b893725806/cf2 in testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806. 2023-02-08 23:55:45,131 INFO [RS:0;jenkins-hbase4:36713-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 2d53e24f43c86247fa3362b893725806/cf1 in testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806. 2023-02-08 23:55:45,131 INFO [RS:0;jenkins-hbase4:36713-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-e5c43717a62e4fd1887e4599baf22d99, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e3091e07c2b4629ae6f968ffb59fee2, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e099f82c64f4d3fa4ec030b486b560a, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-a237c358ab424b3eb42d514292a522b2, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-c14da944b6fa4d8f894bf0da679a9cb0, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-f830532c24074658900c91541721fce8, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-dbcf14fcf1d94353a8f2aa3f47cd1045] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/.tmp, totalSize=34.9 K 2023-02-08 23:55:45,131 INFO [RS:0;jenkins-hbase4:36713-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-787b4e1a353d479484868379d912727c, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-3b2a7b13dd1b489ca09de59cc423fece, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-44c787fdaba3472382b81588f062dcb5, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-10153ec5089d46a7bb2b66cb21fdc5bb, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-2576c52ed53c41548ab0552080039fce, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-1f4d5e17c0ba4488a52526f12c647534, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-449179b3fd5149809b3583f312303ec3] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/.tmp, totalSize=34.9 K 2023-02-08 23:55:45,132 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-e5c43717a62e4fd1887e4599baf22d99, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900543387 2023-02-08 23:55:45,132 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-787b4e1a353d479484868379d912727c, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900543387 2023-02-08 23:55:45,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806. 2023-02-08 23:55:45,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806. 2023-02-08 23:55:45,135 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=2d53e24f43c86247fa3362b893725806, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:45,135 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900545135"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900545135"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900545135"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900545135"}]},"ts":"1675900545135"} 2023-02-08 23:55:45,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-08 23:55:45,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 2d53e24f43c86247fa3362b893725806, server=jenkins-hbase4.apache.org,36713,1675900441074 in 221 msec 2023-02-08 23:55:45,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-02-08 23:55:45,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2d53e24f43c86247fa3362b893725806, ASSIGN in 381 msec 2023-02-08 23:55:45,141 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e3091e07c2b4629ae6f968ffb59fee2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900543553 2023-02-08 23:55:45,141 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-3b2a7b13dd1b489ca09de59cc423fece, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900543553 2023-02-08 23:55:45,141 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e099f82c64f4d3fa4ec030b486b560a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900543723 2023-02-08 23:55:45,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=5ff780cedf338f2c445fc440d550d8a2, daughterA=2d53e24f43c86247fa3362b893725806, daughterB=338ee863654205b6854621d67961edb9 in 691 msec 2023-02-08 23:55:45,142 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-44c787fdaba3472382b81588f062dcb5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900543723 2023-02-08 23:55:45,142 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-a237c358ab424b3eb42d514292a522b2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900543888 2023-02-08 23:55:45,142 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-10153ec5089d46a7bb2b66cb21fdc5bb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900543888 2023-02-08 23:55:45,142 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-c14da944b6fa4d8f894bf0da679a9cb0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900544065 2023-02-08 23:55:45,142 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-2576c52ed53c41548ab0552080039fce, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900544065 2023-02-08 23:55:45,143 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-f830532c24074658900c91541721fce8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900544269 2023-02-08 23:55:45,143 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-1f4d5e17c0ba4488a52526f12c647534, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900544269 2023-02-08 23:55:45,143 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-dbcf14fcf1d94353a8f2aa3f47cd1045, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675900544443 2023-02-08 23:55:45,143 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-449179b3fd5149809b3583f312303ec3, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675900544443 2023-02-08 23:55:45,172 INFO [RS:0;jenkins-hbase4:36713-longCompactions-0] throttle.PressureAwareThroughputController(145): 2d53e24f43c86247fa3362b893725806#cf2#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:45,172 INFO [RS:0;jenkins-hbase4:36713-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2d53e24f43c86247fa3362b893725806#cf1#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:45,182 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/.tmp/cf1/a14a74b24df84fcdacd2a43f30c6b9bb as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/a14a74b24df84fcdacd2a43f30c6b9bb 2023-02-08 23:55:45,186 INFO [RS:0;jenkins-hbase4:36713-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2d53e24f43c86247fa3362b893725806/cf1 of 2d53e24f43c86247fa3362b893725806 into a14a74b24df84fcdacd2a43f30c6b9bb(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:45,186 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2d53e24f43c86247fa3362b893725806: 2023-02-08 23:55:45,186 INFO [RS:0;jenkins-hbase4:36713-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806., storeName=2d53e24f43c86247fa3362b893725806/cf1, priority=9, startTime=1675900545130; duration=0sec 2023-02-08 23:55:45,186 DEBUG [RS:0;jenkins-hbase4:36713-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:45,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36713] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:43382 deadline: 1675900605200, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2. is not online on jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:45,318 DEBUG [Listener at localhost/37973] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-08 23:55:45,321 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:55:45,322 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:45,322 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:45,323 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:45,323 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:45,323 INFO [Listener at localhost/37973] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-08 23:55:45,323 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-08 23:55:45,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 23:55:45,325 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:45,326 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:45,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-08 23:55:45,326 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900545326"}]},"ts":"1675900545326"} 2023-02-08 23:55:45,327 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-08 23:55:45,329 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-08 23:55:45,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2d53e24f43c86247fa3362b893725806, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=338ee863654205b6854621d67961edb9, UNASSIGN}] 2023-02-08 23:55:45,330 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=338ee863654205b6854621d67961edb9, UNASSIGN 2023-02-08 23:55:45,330 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=338ee863654205b6854621d67961edb9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:45,331 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900545330"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900545330"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900545330"}]},"ts":"1675900545330"} 2023-02-08 23:55:45,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 338ee863654205b6854621d67961edb9, server=jenkins-hbase4.apache.org,36713,1675900441074}] 2023-02-08 23:55:45,333 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2d53e24f43c86247fa3362b893725806, UNASSIGN 2023-02-08 23:55:45,334 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=2d53e24f43c86247fa3362b893725806, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:45,334 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900545334"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900545334"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900545334"}]},"ts":"1675900545334"} 2023-02-08 23:55:45,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 2d53e24f43c86247fa3362b893725806, server=jenkins-hbase4.apache.org,36713,1675900441074}] 2023-02-08 23:55:45,426 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:45,427 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:45,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-08 23:55:45,483 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 338ee863654205b6854621d67961edb9 2023-02-08 23:55:45,484 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 338ee863654205b6854621d67961edb9, disabling compactions & flushes 2023-02-08 23:55:45,484 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9. 2023-02-08 23:55:45,484 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9. 2023-02-08 23:55:45,484 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9. after waiting 0 ms 2023-02-08 23:55:45,484 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9. 2023-02-08 23:55:45,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-08 23:55:45,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:45,488 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9. 2023-02-08 23:55:45,488 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 338ee863654205b6854621d67961edb9: Waiting for close lock at 1675900545483Running coprocessor pre-close hooks at 1675900545483Disabling compacts and flushes for region at 1675900545483Disabling writes for close at 1675900545484 (+1 ms)Writing region close event to WAL at 1675900545485 (+1 ms)Running coprocessor post-close hooks at 1675900545487 (+2 ms)Closed at 1675900545488 (+1 ms) 2023-02-08 23:55:45,489 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 338ee863654205b6854621d67961edb9 2023-02-08 23:55:45,489 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2d53e24f43c86247fa3362b893725806 2023-02-08 23:55:45,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2d53e24f43c86247fa3362b893725806, disabling compactions & flushes 2023-02-08 23:55:45,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806. 2023-02-08 23:55:45,489 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=338ee863654205b6854621d67961edb9, regionState=CLOSED 2023-02-08 23:55:45,489 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900545489"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900545489"}]},"ts":"1675900545489"} 2023-02-08 23:55:45,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-08 23:55:45,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 338ee863654205b6854621d67961edb9, server=jenkins-hbase4.apache.org,36713,1675900441074 in 159 msec 2023-02-08 23:55:45,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=338ee863654205b6854621d67961edb9, UNASSIGN in 163 msec 2023-02-08 23:55:45,583 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/.tmp/cf2/2992c5cba52e43fb9e192ff3f2e7f8cc as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/2992c5cba52e43fb9e192ff3f2e7f8cc 2023-02-08 23:55:45,587 INFO [RS:0;jenkins-hbase4:36713-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2d53e24f43c86247fa3362b893725806/cf2 of 2d53e24f43c86247fa3362b893725806 into 2992c5cba52e43fb9e192ff3f2e7f8cc(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:45,587 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2d53e24f43c86247fa3362b893725806: 2023-02-08 23:55:45,587 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806. 2023-02-08 23:55:45,587 INFO [RS:0;jenkins-hbase4:36713-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806., storeName=2d53e24f43c86247fa3362b893725806/cf2, priority=9, startTime=1675900545130; duration=0sec 2023-02-08 23:55:45,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806. 2023-02-08 23:55:45,587 DEBUG [RS:0;jenkins-hbase4:36713-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:45,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806. after waiting 0 ms 2023-02-08 23:55:45,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806. 2023-02-08 23:55:45,588 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-787b4e1a353d479484868379d912727c, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-3b2a7b13dd1b489ca09de59cc423fece, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-44c787fdaba3472382b81588f062dcb5, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-10153ec5089d46a7bb2b66cb21fdc5bb, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-2576c52ed53c41548ab0552080039fce, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-1f4d5e17c0ba4488a52526f12c647534, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-449179b3fd5149809b3583f312303ec3] to archive 2023-02-08 23:55:45,589 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:45,590 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-787b4e1a353d479484868379d912727c to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-787b4e1a353d479484868379d912727c 2023-02-08 23:55:45,591 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-3b2a7b13dd1b489ca09de59cc423fece to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-3b2a7b13dd1b489ca09de59cc423fece 2023-02-08 23:55:45,593 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-44c787fdaba3472382b81588f062dcb5 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-44c787fdaba3472382b81588f062dcb5 2023-02-08 23:55:45,594 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-10153ec5089d46a7bb2b66cb21fdc5bb to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-10153ec5089d46a7bb2b66cb21fdc5bb 2023-02-08 23:55:45,595 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-2576c52ed53c41548ab0552080039fce to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-2576c52ed53c41548ab0552080039fce 2023-02-08 23:55:45,595 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-1f4d5e17c0ba4488a52526f12c647534 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-1f4d5e17c0ba4488a52526f12c647534 2023-02-08 23:55:45,596 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-449179b3fd5149809b3583f312303ec3 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-449179b3fd5149809b3583f312303ec3 2023-02-08 23:55:45,598 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-e5c43717a62e4fd1887e4599baf22d99, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e3091e07c2b4629ae6f968ffb59fee2, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e099f82c64f4d3fa4ec030b486b560a, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-a237c358ab424b3eb42d514292a522b2, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-c14da944b6fa4d8f894bf0da679a9cb0, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-f830532c24074658900c91541721fce8, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-dbcf14fcf1d94353a8f2aa3f47cd1045] to archive 2023-02-08 23:55:45,598 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:45,599 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-e5c43717a62e4fd1887e4599baf22d99 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-e5c43717a62e4fd1887e4599baf22d99 2023-02-08 23:55:45,600 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e3091e07c2b4629ae6f968ffb59fee2 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e3091e07c2b4629ae6f968ffb59fee2 2023-02-08 23:55:45,601 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e099f82c64f4d3fa4ec030b486b560a to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-4e099f82c64f4d3fa4ec030b486b560a 2023-02-08 23:55:45,602 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-a237c358ab424b3eb42d514292a522b2 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-a237c358ab424b3eb42d514292a522b2 2023-02-08 23:55:45,603 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-c14da944b6fa4d8f894bf0da679a9cb0 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-c14da944b6fa4d8f894bf0da679a9cb0 2023-02-08 23:55:45,603 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-f830532c24074658900c91541721fce8 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-f830532c24074658900c91541721fce8 2023-02-08 23:55:45,604 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-dbcf14fcf1d94353a8f2aa3f47cd1045 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/testSplitTableRegionEmptyDaughter=5ff780cedf338f2c445fc440d550d8a2-dbcf14fcf1d94353a8f2aa3f47cd1045 2023-02-08 23:55:45,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 23:55:45,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:45,608 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806. 2023-02-08 23:55:45,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2d53e24f43c86247fa3362b893725806: Waiting for close lock at 1675900545489Running coprocessor pre-close hooks at 1675900545489Disabling compacts and flushes for region at 1675900545489Disabling writes for close at 1675900545587 (+98 ms)Writing region close event to WAL at 1675900545605 (+18 ms)Running coprocessor post-close hooks at 1675900545608 (+3 ms)Closed at 1675900545608 2023-02-08 23:55:45,609 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2d53e24f43c86247fa3362b893725806 2023-02-08 23:55:45,609 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=2d53e24f43c86247fa3362b893725806, regionState=CLOSED 2023-02-08 23:55:45,610 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900545609"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900545609"}]},"ts":"1675900545609"} 2023-02-08 23:55:45,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-08 23:55:45,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 2d53e24f43c86247fa3362b893725806, server=jenkins-hbase4.apache.org,36713,1675900441074 in 275 msec 2023-02-08 23:55:45,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-02-08 23:55:45,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2d53e24f43c86247fa3362b893725806, UNASSIGN in 283 msec 2023-02-08 23:55:45,614 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900545614"}]},"ts":"1675900545614"} 2023-02-08 23:55:45,614 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-08 23:55:45,616 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-08 23:55:45,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 293 msec 2023-02-08 23:55:45,627 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:45,628 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:45,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-08 23:55:45,629 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-08 23:55:45,629 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:45,629 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:45,629 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-08 23:55:45,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 23:55:45,631 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 23:55:45,631 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:45,632 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:45,632 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 23:55:45,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-08 23:55:45,636 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:45,636 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806 2023-02-08 23:55:45,636 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9 2023-02-08 23:55:45,637 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9/recovered.edits] 2023-02-08 23:55:45,637 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/recovered.edits] 2023-02-08 23:55:45,638 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/recovered.edits] 2023-02-08 23:55:45,640 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/a14a74b24df84fcdacd2a43f30c6b9bb to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf1/a14a74b24df84fcdacd2a43f30c6b9bb 2023-02-08 23:55:45,640 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/10153ec5089d46a7bb2b66cb21fdc5bb to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/10153ec5089d46a7bb2b66cb21fdc5bb 2023-02-08 23:55:45,642 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/1f4d5e17c0ba4488a52526f12c647534 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/1f4d5e17c0ba4488a52526f12c647534 2023-02-08 23:55:45,643 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/2992c5cba52e43fb9e192ff3f2e7f8cc to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/cf2/2992c5cba52e43fb9e192ff3f2e7f8cc 2023-02-08 23:55:45,643 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/2576c52ed53c41548ab0552080039fce to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/2576c52ed53c41548ab0552080039fce 2023-02-08 23:55:45,643 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9/recovered.edits/88.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9/recovered.edits/88.seqid 2023-02-08 23:55:45,644 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/338ee863654205b6854621d67961edb9 2023-02-08 23:55:45,644 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/3b2a7b13dd1b489ca09de59cc423fece to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/3b2a7b13dd1b489ca09de59cc423fece 2023-02-08 23:55:45,645 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/449179b3fd5149809b3583f312303ec3 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/449179b3fd5149809b3583f312303ec3 2023-02-08 23:55:45,645 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/recovered.edits/90.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806/recovered.edits/90.seqid 2023-02-08 23:55:45,646 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/2d53e24f43c86247fa3362b893725806 2023-02-08 23:55:45,646 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/44c787fdaba3472382b81588f062dcb5 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/44c787fdaba3472382b81588f062dcb5 2023-02-08 23:55:45,647 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/787b4e1a353d479484868379d912727c to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf1/787b4e1a353d479484868379d912727c 2023-02-08 23:55:45,649 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e099f82c64f4d3fa4ec030b486b560a to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e099f82c64f4d3fa4ec030b486b560a 2023-02-08 23:55:45,650 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e3091e07c2b4629ae6f968ffb59fee2 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/4e3091e07c2b4629ae6f968ffb59fee2 2023-02-08 23:55:45,651 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/a237c358ab424b3eb42d514292a522b2 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/a237c358ab424b3eb42d514292a522b2 2023-02-08 23:55:45,652 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/c14da944b6fa4d8f894bf0da679a9cb0 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/c14da944b6fa4d8f894bf0da679a9cb0 2023-02-08 23:55:45,653 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/dbcf14fcf1d94353a8f2aa3f47cd1045 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/dbcf14fcf1d94353a8f2aa3f47cd1045 2023-02-08 23:55:45,653 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/e5c43717a62e4fd1887e4599baf22d99 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/e5c43717a62e4fd1887e4599baf22d99 2023-02-08 23:55:45,654 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/f830532c24074658900c91541721fce8 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/cf2/f830532c24074658900c91541721fce8 2023-02-08 23:55:45,656 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/recovered.edits/85.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2/recovered.edits/85.seqid 2023-02-08 23:55:45,657 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegionEmptyDaughter/5ff780cedf338f2c445fc440d550d8a2 2023-02-08 23:55:45,657 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-08 23:55:45,658 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 23:55:45,660 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-08 23:55:45,662 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-08 23:55:45,662 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 23:55:45,663 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-08 23:55:45,663 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900545663"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:45,663 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900545663"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:45,663 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900545663"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:45,664 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 23:55:45,664 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 5ff780cedf338f2c445fc440d550d8a2, NAME => 'testSplitTableRegionEmptyDaughter,,1675900542880.5ff780cedf338f2c445fc440d550d8a2.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 2d53e24f43c86247fa3362b893725806, NAME => 'testSplitTableRegionEmptyDaughter,,1675900544449.2d53e24f43c86247fa3362b893725806.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 338ee863654205b6854621d67961edb9, NAME => 'testSplitTableRegionEmptyDaughter,71,1675900544449.338ee863654205b6854621d67961edb9.', STARTKEY => '71', ENDKEY => ''}] 2023-02-08 23:55:45,664 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-08 23:55:45,664 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675900545664"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:45,665 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-08 23:55:45,667 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 23:55:45,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 38 msec 2023-02-08 23:55:45,732 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:45,733 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:45,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-08 23:55:45,733 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-08 23:55:45,747 INFO [Listener at localhost/37973] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375 (was 372) - Thread LEAK? -, OpenFileDescriptor=830 (was 828) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=109 (was 109), ProcessCount=168 (was 168), AvailableMemoryMB=3053 (was 3059) 2023-02-08 23:55:45,760 INFO [Listener at localhost/37973] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375, OpenFileDescriptor=830, MaxFileDescriptor=60000, SystemLoadAverage=109, ProcessCount=168, AvailableMemoryMB=3052 2023-02-08 23:55:45,760 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:55:45,760 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:45,760 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:45,763 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 23:55:45,764 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 23:55:45,765 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-08 23:55:45,766 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:55:45,767 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:55:45,768 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:45,768 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696 empty. 2023-02-08 23:55:45,768 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:45,768 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-08 23:55:45,779 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-08 23:55:45,780 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4054f5078a49e39db8d3b092f3d40696, NAME => 'testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp 2023-02-08 23:55:46,189 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:46,190 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 4054f5078a49e39db8d3b092f3d40696, disabling compactions & flushes 2023-02-08 23:55:46,190 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:46,190 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:46,190 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. after waiting 0 ms 2023-02-08 23:55:46,190 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:46,190 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:46,190 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 4054f5078a49e39db8d3b092f3d40696: Waiting for close lock at 1675900546190Disabling compacts and flushes for region at 1675900546190Disabling writes for close at 1675900546190Writing region close event to WAL at 1675900546190Closed at 1675900546190 2023-02-08 23:55:46,192 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:55:46,192 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900546192"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900546192"}]},"ts":"1675900546192"} 2023-02-08 23:55:46,193 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:55:46,194 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:55:46,194 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900546194"}]},"ts":"1675900546194"} 2023-02-08 23:55:46,195 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-08 23:55:46,197 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 23:55:46,198 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 23:55:46,198 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 23:55:46,198 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 23:55:46,198 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 23:55:46,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=4054f5078a49e39db8d3b092f3d40696, ASSIGN}] 2023-02-08 23:55:46,199 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=4054f5078a49e39db8d3b092f3d40696, ASSIGN 2023-02-08 23:55:46,200 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=4054f5078a49e39db8d3b092f3d40696, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42343,1675900441151; forceNewPlan=false, retain=false 2023-02-08 23:55:46,350 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 23:55:46,351 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=4054f5078a49e39db8d3b092f3d40696, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,351 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900546351"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900546351"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900546351"}]},"ts":"1675900546351"} 2023-02-08 23:55:46,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 4054f5078a49e39db8d3b092f3d40696, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:46,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:46,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4054f5078a49e39db8d3b092f3d40696, NAME => 'testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:55:46,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:55:46,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:46,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:46,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:46,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:46,508 INFO [StoreOpener-4054f5078a49e39db8d3b092f3d40696-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:46,509 DEBUG [StoreOpener-4054f5078a49e39db8d3b092f3d40696-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1 2023-02-08 23:55:46,509 DEBUG [StoreOpener-4054f5078a49e39db8d3b092f3d40696-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1 2023-02-08 23:55:46,509 INFO [StoreOpener-4054f5078a49e39db8d3b092f3d40696-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4054f5078a49e39db8d3b092f3d40696 columnFamilyName cf1 2023-02-08 23:55:46,510 INFO [StoreOpener-4054f5078a49e39db8d3b092f3d40696-1] regionserver.HStore(310): Store=4054f5078a49e39db8d3b092f3d40696/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:46,510 INFO [StoreOpener-4054f5078a49e39db8d3b092f3d40696-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:46,511 DEBUG [StoreOpener-4054f5078a49e39db8d3b092f3d40696-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2 2023-02-08 23:55:46,511 DEBUG [StoreOpener-4054f5078a49e39db8d3b092f3d40696-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2 2023-02-08 23:55:46,511 INFO [StoreOpener-4054f5078a49e39db8d3b092f3d40696-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4054f5078a49e39db8d3b092f3d40696 columnFamilyName cf2 2023-02-08 23:55:46,512 INFO [StoreOpener-4054f5078a49e39db8d3b092f3d40696-1] regionserver.HStore(310): Store=4054f5078a49e39db8d3b092f3d40696/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:46,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:46,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:46,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:46,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:46,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:55:46,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4054f5078a49e39db8d3b092f3d40696; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74557072, jitterRate=0.11098694801330566}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:46,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4054f5078a49e39db8d3b092f3d40696: Running coprocessor pre-open hook at 1675900546507Writing region info on filesystem at 1675900546507Initializing all the Stores at 1675900546508 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900546508Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900546508Cleaning up temporary data from old regions at 1675900546513 (+5 ms)Cleaning up detritus from prior splits at 1675900546513Running coprocessor post-open hooks at 1675900546516 (+3 ms)Region opened successfully at 1675900546516 2023-02-08 23:55:46,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696., pid=143, masterSystemTime=1675900546503 2023-02-08 23:55:46,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:46,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:46,518 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=4054f5078a49e39db8d3b092f3d40696, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,518 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900546518"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900546518"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900546518"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900546518"}]},"ts":"1675900546518"} 2023-02-08 23:55:46,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-08 23:55:46,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 4054f5078a49e39db8d3b092f3d40696, server=jenkins-hbase4.apache.org,42343,1675900441151 in 167 msec 2023-02-08 23:55:46,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-08 23:55:46,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=4054f5078a49e39db8d3b092f3d40696, ASSIGN in 323 msec 2023-02-08 23:55:46,522 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:55:46,522 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900546522"}]},"ts":"1675900546522"} 2023-02-08 23:55:46,523 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-08 23:55:46,525 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:55:46,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 761 msec 2023-02-08 23:55:46,771 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:46,771 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:46,771 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:46,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 23:55:46,773 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 23:55:46,773 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:46,773 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 23:55:46,773 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 23:55:46,773 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,773 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,775 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:46,775 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:46,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:46,775 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,775 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:46,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:46,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:46,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:46,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:46,776 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,776 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:46,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:46,777 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:46,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:46,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:46,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:46,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:46,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:46,778 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:46,778 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:46,778 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. started... 2023-02-08 23:55:46,779 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4054f5078a49e39db8d3b092f3d40696 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 23:55:46,780 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,781 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,781 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,781 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:46,781 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,781 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:46,790 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/821c425d656547748d94125931a7da72 2023-02-08 23:55:46,801 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/9f1040e336e340c7a051cc3d623fc214 2023-02-08 23:55:46,804 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/821c425d656547748d94125931a7da72 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/821c425d656547748d94125931a7da72 2023-02-08 23:55:46,808 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/821c425d656547748d94125931a7da72, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:55:46,808 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/9f1040e336e340c7a051cc3d623fc214 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/9f1040e336e340c7a051cc3d623fc214 2023-02-08 23:55:46,811 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/9f1040e336e340c7a051cc3d623fc214, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:55:46,812 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 4054f5078a49e39db8d3b092f3d40696 in 33ms, sequenceid=6, compaction requested=false 2023-02-08 23:55:46,812 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-08 23:55:46,813 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4054f5078a49e39db8d3b092f3d40696: 2023-02-08 23:55:46,813 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:46,813 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:46,813 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:46,813 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,813 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:46,813 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:46,815 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:46,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:46,815 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:46,815 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,815 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:46,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:46,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:46,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:46,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:46,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:46,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 23:55:46,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34c32d93[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:46,817 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 23:55:46,818 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,820 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,820 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,820 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:46,820 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:46,820 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,821 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,821 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:46,821 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:46,821 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:46,821 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:46,820 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:46,820 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:46,821 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:46,821 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:46,823 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,823 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:46,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,823 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:46,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:46,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:46,823 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:46,826 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:46,826 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:46,826 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:46,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:46,826 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:46,826 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:46,826 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:46,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:46,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:46,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:46,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:46,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:46,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:46,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 23:55:46,829 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 23:55:46,829 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:46,829 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:46,829 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 23:55:46,829 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:46,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:46,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:46,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,832 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:46,832 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:46,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:46,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:46,832 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:46,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:46,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:46,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,833 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:46,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,834 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:46,834 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:46,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:46,835 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:46,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:46,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:46,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:46,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:46,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:46,837 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:46,837 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:46,837 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:46,837 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:46,837 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:46,837 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:46,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:46,837 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:46,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:46,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:46,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:46,837 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:46,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:46,837 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 23:55:46,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:46,837 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:46,837 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:46,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:46,838 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,837 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:46,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:46,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegion because node does not exist (not an error) 2023-02-08 23:55:46,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:46,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:46,838 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,838 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:46,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:46,838 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:46,838 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:46,838 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,838 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:46,838 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,838 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,838 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,938 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:46,938 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:46,938 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:46,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:46,948 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:46,948 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:46,948 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:46,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:46,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 23:55:46,950 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 23:55:46,950 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:46,950 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 23:55:46,950 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 23:55:46,950 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,950 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,951 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:46,951 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,951 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:46,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:46,951 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:46,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:46,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:46,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:46,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:46,951 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,952 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:46,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:46,952 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:46,953 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:46,953 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:46,953 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:46,953 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:46,953 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:46,953 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:46,953 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:46,954 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:46,954 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:46,954 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. started... 2023-02-08 23:55:46,954 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4054f5078a49e39db8d3b092f3d40696 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:46,954 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,956 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,956 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:46,956 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,957 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,957 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:46,970 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/08fde390952f42419bbf13efd9ac5222 2023-02-08 23:55:46,981 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/055d1255347340e4a0d4b58e09ad9d08 2023-02-08 23:55:46,984 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/08fde390952f42419bbf13efd9ac5222 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/08fde390952f42419bbf13efd9ac5222 2023-02-08 23:55:46,987 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/08fde390952f42419bbf13efd9ac5222, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:55:46,988 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/055d1255347340e4a0d4b58e09ad9d08 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/055d1255347340e4a0d4b58e09ad9d08 2023-02-08 23:55:46,991 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/055d1255347340e4a0d4b58e09ad9d08, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 23:55:46,991 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4054f5078a49e39db8d3b092f3d40696 in 37ms, sequenceid=19, compaction requested=false 2023-02-08 23:55:46,991 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4054f5078a49e39db8d3b092f3d40696: 2023-02-08 23:55:46,991 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:46,991 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:46,991 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:46,991 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,991 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:46,992 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:46,993 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,993 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:46,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:46,993 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,993 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:46,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:46,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:46,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:46,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:46,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:46,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:46,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 23:55:46,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6de1b026[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:46,995 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 23:55:46,995 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,996 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,996 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,996 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,996 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:46,996 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:46,996 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:46,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,996 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:46,996 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:46,996 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:46,996 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:46,996 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:46,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:46,997 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:46,997 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:46,997 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:46,997 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:46,997 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:46,998 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:46,999 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:46,999 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:46,999 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,000 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,000 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,000 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,000 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:47,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 23:55:47,003 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 23:55:47,003 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:47,003 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:47,003 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 23:55:47,003 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:47,004 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,004 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,004 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,005 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,005 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,005 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,005 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,005 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,005 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,006 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,007 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,007 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,008 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,010 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,010 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,010 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,010 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 23:55:47,010 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:47,010 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,010 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,010 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,010 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,010 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,010 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,010 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:47,010 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,011 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:47,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,011 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,011 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,011 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,011 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,011 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,011 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,011 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,011 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,111 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:47,111 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:47,111 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:47,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:47,121 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:47,121 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:47,121 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:47,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:47,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 23:55:47,123 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 23:55:47,123 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,123 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 23:55:47,123 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 23:55:47,123 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,123 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,125 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,125 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,125 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,125 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,126 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,126 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:47,127 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,127 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:47,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:47,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:47,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:47,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:47,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,127 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:47,127 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. started... 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:47,127 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:47,128 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4054f5078a49e39db8d3b092f3d40696 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:47,136 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,136 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,137 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,137 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,137 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,137 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,141 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/627bf257c34a456abe4f2fe447dcef6c 2023-02-08 23:55:47,154 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/34b97c424a564591b7673a3970dea52a 2023-02-08 23:55:47,157 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/627bf257c34a456abe4f2fe447dcef6c as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/627bf257c34a456abe4f2fe447dcef6c 2023-02-08 23:55:47,160 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/627bf257c34a456abe4f2fe447dcef6c, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:55:47,161 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/34b97c424a564591b7673a3970dea52a as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/34b97c424a564591b7673a3970dea52a 2023-02-08 23:55:47,164 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/34b97c424a564591b7673a3970dea52a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 23:55:47,165 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4054f5078a49e39db8d3b092f3d40696 in 37ms, sequenceid=32, compaction requested=true 2023-02-08 23:55:47,165 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4054f5078a49e39db8d3b092f3d40696: 2023-02-08 23:55:47,165 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:47,165 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:47,165 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:47,165 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,165 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:47,165 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:47,166 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,166 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,166 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,166 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 23:55:47,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@234787d7[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:47,168 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 23:55:47,168 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,170 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,170 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,170 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:47,170 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,170 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:47,170 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,170 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:47,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,170 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:47,170 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:47,170 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:47,170 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:47,170 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:47,170 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:47,170 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:47,171 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,171 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,171 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,171 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,171 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,171 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,172 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,171 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,171 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,172 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:47,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 23:55:47,174 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 23:55:47,174 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:47,174 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:47,174 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 23:55:47,174 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:47,175 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,175 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,176 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,176 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,175 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,175 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,176 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,176 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,177 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,177 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,178 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,179 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,179 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,181 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,181 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,181 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,181 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,181 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,181 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 23:55:47,181 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:47,181 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,181 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,181 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,182 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,182 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,182 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,182 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:47,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,181 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,182 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:47,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,182 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,182 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,182 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,182 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,182 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,182 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,282 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:47,282 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:47,282 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:47,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:47,292 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:47,292 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:47,292 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:47,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:47,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 23:55:47,294 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 23:55:47,294 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,294 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 23:55:47,294 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 23:55:47,294 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,294 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,296 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,296 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,296 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,296 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,297 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,297 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,297 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,297 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:47,298 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:47,298 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:47,298 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:47,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:47,299 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:47,299 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. started... 2023-02-08 23:55:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:47,299 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4054f5078a49e39db8d3b092f3d40696 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:47,300 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,307 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/58a88a5539b34dcc9ffc6697598f6836 2023-02-08 23:55:47,316 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/74d8e28e51354906961ccd1f7f3d6b26 2023-02-08 23:55:47,319 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/58a88a5539b34dcc9ffc6697598f6836 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/58a88a5539b34dcc9ffc6697598f6836 2023-02-08 23:55:47,322 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/58a88a5539b34dcc9ffc6697598f6836, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:55:47,323 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/74d8e28e51354906961ccd1f7f3d6b26 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/74d8e28e51354906961ccd1f7f3d6b26 2023-02-08 23:55:47,326 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/74d8e28e51354906961ccd1f7f3d6b26, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 23:55:47,326 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4054f5078a49e39db8d3b092f3d40696 in 27ms, sequenceid=45, compaction requested=true 2023-02-08 23:55:47,326 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4054f5078a49e39db8d3b092f3d40696: 2023-02-08 23:55:47,326 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:47,326 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:47,326 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:47,326 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,327 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:47,327 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:47,328 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,328 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,329 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,329 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,330 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 23:55:47,330 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d3754b5[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:47,330 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 23:55:47,330 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,332 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,332 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,332 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,332 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:47,332 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:47,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,332 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,333 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:47,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,333 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:47,333 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:47,333 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:47,333 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:47,333 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:47,333 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:47,333 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:47,334 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,334 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,334 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,334 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,334 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,334 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,334 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,335 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,335 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,335 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:47,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 23:55:47,338 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 23:55:47,338 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:47,338 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:47,338 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 23:55:47,338 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:47,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,340 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,340 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,341 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,342 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,344 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,344 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,344 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,350 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,350 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,350 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,350 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 23:55:47,350 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:47,350 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,350 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,350 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,350 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,351 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:47,351 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:47,350 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,351 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,351 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,351 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,351 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,351 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,351 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,351 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,351 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,351 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,351 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,451 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:47,451 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:47,451 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:47,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:47,460 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:47,460 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:47,461 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:47,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:47,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 23:55:47,462 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 23:55:47,462 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,462 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 23:55:47,462 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 23:55:47,463 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,463 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,464 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,464 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,464 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,464 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,464 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,464 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:47,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,465 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:47,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:47,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:47,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:47,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:47,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:47,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:47,466 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:47,466 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. started... 2023-02-08 23:55:47,467 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4054f5078a49e39db8d3b092f3d40696 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:47,467 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,468 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,468 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,468 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,468 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,468 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,475 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/421b21144eaa4642a6a495936083ed99 2023-02-08 23:55:47,485 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/583011af5c004bccb137685cb2eeae1f 2023-02-08 23:55:47,488 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/421b21144eaa4642a6a495936083ed99 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/421b21144eaa4642a6a495936083ed99 2023-02-08 23:55:47,491 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/421b21144eaa4642a6a495936083ed99, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:55:47,492 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/583011af5c004bccb137685cb2eeae1f as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/583011af5c004bccb137685cb2eeae1f 2023-02-08 23:55:47,494 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/583011af5c004bccb137685cb2eeae1f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 23:55:47,495 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4054f5078a49e39db8d3b092f3d40696 in 28ms, sequenceid=58, compaction requested=true 2023-02-08 23:55:47,495 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4054f5078a49e39db8d3b092f3d40696: 2023-02-08 23:55:47,495 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:47,495 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:47,495 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:47,495 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,495 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:47,495 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:47,498 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,498 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,498 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,498 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,500 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 23:55:47,500 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30355d35[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:47,500 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 23:55:47,500 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,501 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,501 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,501 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,501 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,501 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:47,501 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:47,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,501 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:47,501 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:47,501 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:47,501 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:47,501 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:47,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,501 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:47,502 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:47,502 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:47,502 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,502 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,502 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,503 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,503 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,503 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,503 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,504 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,504 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,504 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:47,507 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 23:55:47,507 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 23:55:47,507 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:47,507 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:47,507 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 23:55:47,507 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:47,508 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,508 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,508 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,508 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,508 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,508 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,509 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,508 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,509 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,510 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,511 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,511 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,511 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,514 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,514 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,514 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,514 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,514 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,514 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,514 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,514 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,514 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 23:55:47,514 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:47,514 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,514 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,515 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:47,514 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,515 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:47,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,515 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,515 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,515 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,515 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,515 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,515 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,515 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,615 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:47,615 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:47,615 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:47,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:47,625 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:47,626 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:47,626 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:47,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:47,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 23:55:47,627 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 23:55:47,628 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,628 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 23:55:47,628 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 23:55:47,628 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,628 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,629 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,629 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,629 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,629 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,630 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,630 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:47,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,631 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:47,631 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:47,631 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,631 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:47,631 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 23:55:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36199,1675900441120' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:47,632 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:47,632 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:47,632 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 23:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36713,1675900441074' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:47,632 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 23:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 23:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 23:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:47,633 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:47,633 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:47,633 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. started... 2023-02-08 23:55:47,633 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4054f5078a49e39db8d3b092f3d40696 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 23:55:47,634 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,634 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,634 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,635 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,635 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,635 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,642 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/4a61708f2c5e4cffa1af06883be293be 2023-02-08 23:55:47,652 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/bd55bec7511545388f8678d9ef2f5ffa 2023-02-08 23:55:47,656 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/4a61708f2c5e4cffa1af06883be293be as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/4a61708f2c5e4cffa1af06883be293be 2023-02-08 23:55:47,660 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/4a61708f2c5e4cffa1af06883be293be, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:55:47,661 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/bd55bec7511545388f8678d9ef2f5ffa as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/bd55bec7511545388f8678d9ef2f5ffa 2023-02-08 23:55:47,664 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/bd55bec7511545388f8678d9ef2f5ffa, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 23:55:47,664 INFO [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4054f5078a49e39db8d3b092f3d40696 in 31ms, sequenceid=71, compaction requested=true 2023-02-08 23:55:47,664 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4054f5078a49e39db8d3b092f3d40696: 2023-02-08 23:55:47,664 DEBUG [rs(jenkins-hbase4.apache.org,42343,1675900441151)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:47,664 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:47,665 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:47,665 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,665 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 23:55:47,665 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 23:55:47,666 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,666 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,666 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,666 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42343,1675900441151' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 23:55:47,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50b5bb95[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:47,668 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 23:55:47,668 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,669 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,669 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,669 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,669 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:47,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,669 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,669 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:47,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,669 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:47,670 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:47,670 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:47,670 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36713,1675900441074' in zk 2023-02-08 23:55:47,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,670 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36199,1675900441120' in zk 2023-02-08 23:55:47,670 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 23:55:47,670 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 23:55:47,670 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42343,1675900441151' in zk 2023-02-08 23:55:47,679 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,679 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,679 DEBUG [member: 'jenkins-hbase4.apache.org,36713,1675900441074' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,680 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,680 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 23:55:47,680 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,680 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,680 DEBUG [member: 'jenkins-hbase4.apache.org,42343,1675900441151' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,680 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,680 DEBUG [member: 'jenkins-hbase4.apache.org,36199,1675900441120' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 23:55:47,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42343,1675900441151': 2023-02-08 23:55:47,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42343,1675900441151' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 23:55:47,683 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 23:55:47,683 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:47,683 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:47,683 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 23:55:47,683 INFO [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:47,684 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,684 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,684 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,684 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,684 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,684 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,684 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,685 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,685 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,685 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,687 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,687 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 23:55:47,687 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,693 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,693 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,693 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,693 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,694 DEBUG [(jenkins-hbase4.apache.org,37353,1675900439913)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,693 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,694 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 23:55:47,694 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:47,694 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,694 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,694 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 23:55:47,694 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,694 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:47,694 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,694 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,695 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 23:55:47,695 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:47,695 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,695 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:47,695 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,695 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 23:55:47,695 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 23:55:47,794 DEBUG [Listener at localhost/37973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:47,795 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:47,795 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:47,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:47,803 DEBUG [Listener at localhost/37973] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,804 DEBUG [Listener at localhost/37973] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=4054f5078a49e39db8d3b092f3d40696, daughterA=4e1fc455ff9d0a5646cc077e20771f7d, daughterB=7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:47,805 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=4054f5078a49e39db8d3b092f3d40696, daughterA=4e1fc455ff9d0a5646cc077e20771f7d, daughterB=7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:47,805 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=4054f5078a49e39db8d3b092f3d40696, daughterA=4e1fc455ff9d0a5646cc077e20771f7d, daughterB=7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:47,805 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=4054f5078a49e39db8d3b092f3d40696, daughterA=4e1fc455ff9d0a5646cc077e20771f7d, daughterB=7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:47,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=4054f5078a49e39db8d3b092f3d40696, UNASSIGN}] 2023-02-08 23:55:47,809 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=4054f5078a49e39db8d3b092f3d40696, UNASSIGN 2023-02-08 23:55:47,809 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=4054f5078a49e39db8d3b092f3d40696, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:47,809 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900547809"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900547809"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900547809"}]},"ts":"1675900547809"} 2023-02-08 23:55:47,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 4054f5078a49e39db8d3b092f3d40696, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:47,962 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:47,962 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4054f5078a49e39db8d3b092f3d40696, disabling compactions & flushes 2023-02-08 23:55:47,962 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:47,962 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:47,962 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. after waiting 0 ms 2023-02-08 23:55:47,962 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:47,963 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 4054f5078a49e39db8d3b092f3d40696 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 23:55:47,976 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/687415f2bb534cc999814729f76eb084 2023-02-08 23:55:47,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/6cb83133f9234c2dad0f78ac1bc7c33a 2023-02-08 23:55:47,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf1/687415f2bb534cc999814729f76eb084 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/687415f2bb534cc999814729f76eb084 2023-02-08 23:55:47,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/687415f2bb534cc999814729f76eb084, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:55:47,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/.tmp/cf2/6cb83133f9234c2dad0f78ac1bc7c33a as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/6cb83133f9234c2dad0f78ac1bc7c33a 2023-02-08 23:55:47,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/6cb83133f9234c2dad0f78ac1bc7c33a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 23:55:47,998 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4054f5078a49e39db8d3b092f3d40696 in 35ms, sequenceid=82, compaction requested=true 2023-02-08 23:55:48,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 23:55:48,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:48,004 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. 2023-02-08 23:55:48,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4054f5078a49e39db8d3b092f3d40696: Waiting for close lock at 1675900547962Running coprocessor pre-close hooks at 1675900547962Disabling compacts and flushes for region at 1675900547962Disabling writes for close at 1675900547962Obtaining lock to block concurrent updates at 1675900547963 (+1 ms)Preparing flush snapshotting stores in 4054f5078a49e39db8d3b092f3d40696 at 1675900547963Finished memstore snapshotting testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675900547963Flushing stores of testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. at 1675900547963Flushing 4054f5078a49e39db8d3b092f3d40696/cf1: creating writer at 1675900547964 (+1 ms)Flushing 4054f5078a49e39db8d3b092f3d40696/cf1: appending metadata at 1675900547966 (+2 ms)Flushing 4054f5078a49e39db8d3b092f3d40696/cf1: closing flushed file at 1675900547966Flushing 4054f5078a49e39db8d3b092f3d40696/cf2: creating writer at 1675900547979 (+13 ms)Flushing 4054f5078a49e39db8d3b092f3d40696/cf2: appending metadata at 1675900547981 (+2 ms)Flushing 4054f5078a49e39db8d3b092f3d40696/cf2: closing flushed file at 1675900547981Flushing 4054f5078a49e39db8d3b092f3d40696/cf1: reopening flushed file at 1675900547991 (+10 ms)Flushing 4054f5078a49e39db8d3b092f3d40696/cf2: reopening flushed file at 1675900547995 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4054f5078a49e39db8d3b092f3d40696 in 35ms, sequenceid=82, compaction requested=true at 1675900547998 (+3 ms)Writing region close event to WAL at 1675900548001 (+3 ms)Running coprocessor post-close hooks at 1675900548004 (+3 ms)Closed at 1675900548004 2023-02-08 23:55:48,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,006 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=4054f5078a49e39db8d3b092f3d40696, regionState=CLOSED 2023-02-08 23:55:48,006 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900548006"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900548006"}]},"ts":"1675900548006"} 2023-02-08 23:55:48,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-08 23:55:48,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 4054f5078a49e39db8d3b092f3d40696, server=jenkins-hbase4.apache.org,42343,1675900441151 in 198 msec 2023-02-08 23:55:48,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-08 23:55:48,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=4054f5078a49e39db8d3b092f3d40696, UNASSIGN in 202 msec 2023-02-08 23:55:48,024 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=4054f5078a49e39db8d3b092f3d40696, threads=14 2023-02-08 23:55:48,024 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/055d1255347340e4a0d4b58e09ad9d08 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,024 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/34b97c424a564591b7673a3970dea52a for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,025 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/583011af5c004bccb137685cb2eeae1f for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,025 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/6cb83133f9234c2dad0f78ac1bc7c33a for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,025 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/74d8e28e51354906961ccd1f7f3d6b26 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,025 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/9f1040e336e340c7a051cc3d623fc214 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,025 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/08fde390952f42419bbf13efd9ac5222 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,025 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/bd55bec7511545388f8678d9ef2f5ffa for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,025 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/4a61708f2c5e4cffa1af06883be293be for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,025 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/421b21144eaa4642a6a495936083ed99 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,054 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/58a88a5539b34dcc9ffc6697598f6836 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,055 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/627bf257c34a456abe4f2fe447dcef6c for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,055 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/821c425d656547748d94125931a7da72 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,055 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/687415f2bb534cc999814729f76eb084 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,200 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/34b97c424a564591b7673a3970dea52a for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,228 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/055d1255347340e4a0d4b58e09ad9d08 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,240 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/821c425d656547748d94125931a7da72 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,243 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/627bf257c34a456abe4f2fe447dcef6c for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,252 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/bd55bec7511545388f8678d9ef2f5ffa for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,262 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/4a61708f2c5e4cffa1af06883be293be for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,263 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/08fde390952f42419bbf13efd9ac5222 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,288 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/687415f2bb534cc999814729f76eb084 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,288 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/58a88a5539b34dcc9ffc6697598f6836 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,294 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/74d8e28e51354906961ccd1f7f3d6b26 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,294 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/6cb83133f9234c2dad0f78ac1bc7c33a for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,294 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/583011af5c004bccb137685cb2eeae1f for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,295 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/421b21144eaa4642a6a495936083ed99 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,695 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/9f1040e336e340c7a051cc3d623fc214 for region: 4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:48,695 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 4054f5078a49e39db8d3b092f3d40696 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 23:55:48,723 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 23:55:48,725 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 23:55:48,726 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675900548726"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675900548726"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675900548726"}]},"ts":"1675900548726"} 2023-02-08 23:55:48,726 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900548726"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900548726"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900548726"}]},"ts":"1675900548726"} 2023-02-08 23:55:48,726 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900548726"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900548726"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900548726"}]},"ts":"1675900548726"} 2023-02-08 23:55:48,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=4e1fc455ff9d0a5646cc077e20771f7d, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=7b1cdafba8d0100c5a0fda504e0f94d9, ASSIGN}] 2023-02-08 23:55:48,729 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=7b1cdafba8d0100c5a0fda504e0f94d9, ASSIGN 2023-02-08 23:55:48,729 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=7b1cdafba8d0100c5a0fda504e0f94d9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42343,1675900441151; forceNewPlan=false, retain=false 2023-02-08 23:55:48,730 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=4e1fc455ff9d0a5646cc077e20771f7d, ASSIGN 2023-02-08 23:55:48,730 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=4e1fc455ff9d0a5646cc077e20771f7d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42343,1675900441151; forceNewPlan=false, retain=false 2023-02-08 23:55:48,788 ERROR [BP-379259138-172.31.14.131-1675900436765 heartbeating to localhost/127.0.0.1:38079] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-379259138-172.31.14.131-1675900436765 (Datanode Uuid c96d9a8d-7822-404e-9335-ec30f9bdbf87) service to localhost/127.0.0.1:38079 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 23:55:48,789 ERROR [BP-379259138-172.31.14.131-1675900436765 heartbeating to localhost/127.0.0.1:38079] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-379259138-172.31.14.131-1675900436765 (Datanode Uuid e53af517-ccea-4786-a9f1-d1d57e9a860e) service to localhost/127.0.0.1:38079 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 23:55:48,880 INFO [jenkins-hbase4:37353] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 23:55:48,881 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=4e1fc455ff9d0a5646cc077e20771f7d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:48,881 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900548881"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900548881"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900548881"}]},"ts":"1675900548881"} 2023-02-08 23:55:48,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 4e1fc455ff9d0a5646cc077e20771f7d, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:48,884 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=7b1cdafba8d0100c5a0fda504e0f94d9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:48,884 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900548884"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900548884"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900548884"}]},"ts":"1675900548884"} 2023-02-08 23:55:48,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 7b1cdafba8d0100c5a0fda504e0f94d9, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:49,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9. 2023-02-08 23:55:49,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7b1cdafba8d0100c5a0fda504e0f94d9, NAME => 'testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.', STARTKEY => '41', ENDKEY => ''} 2023-02-08 23:55:49,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:55:49,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:49,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:49,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:49,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:49,038 INFO [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:49,038 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1 2023-02-08 23:55:49,038 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1 2023-02-08 23:55:49,039 INFO [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7b1cdafba8d0100c5a0fda504e0f94d9 columnFamilyName cf1 2023-02-08 23:55:49,049 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/08fde390952f42419bbf13efd9ac5222.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/08fde390952f42419bbf13efd9ac5222-top 2023-02-08 23:55:49,052 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/421b21144eaa4642a6a495936083ed99.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/421b21144eaa4642a6a495936083ed99-top 2023-02-08 23:55:49,055 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/4a61708f2c5e4cffa1af06883be293be.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/4a61708f2c5e4cffa1af06883be293be-top 2023-02-08 23:55:49,058 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/58a88a5539b34dcc9ffc6697598f6836.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/58a88a5539b34dcc9ffc6697598f6836-top 2023-02-08 23:55:49,061 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/627bf257c34a456abe4f2fe447dcef6c.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/627bf257c34a456abe4f2fe447dcef6c-top 2023-02-08 23:55:49,064 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/687415f2bb534cc999814729f76eb084.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/687415f2bb534cc999814729f76eb084-top 2023-02-08 23:55:49,067 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/821c425d656547748d94125931a7da72.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/821c425d656547748d94125931a7da72-top 2023-02-08 23:55:49,067 INFO [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(310): Store=7b1cdafba8d0100c5a0fda504e0f94d9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:49,067 INFO [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:49,068 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2 2023-02-08 23:55:49,068 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2 2023-02-08 23:55:49,068 INFO [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7b1cdafba8d0100c5a0fda504e0f94d9 columnFamilyName cf2 2023-02-08 23:55:49,078 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/055d1255347340e4a0d4b58e09ad9d08.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/055d1255347340e4a0d4b58e09ad9d08-top 2023-02-08 23:55:49,081 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/34b97c424a564591b7673a3970dea52a.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/34b97c424a564591b7673a3970dea52a-top 2023-02-08 23:55:49,084 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/583011af5c004bccb137685cb2eeae1f.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/583011af5c004bccb137685cb2eeae1f-top 2023-02-08 23:55:49,086 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/6cb83133f9234c2dad0f78ac1bc7c33a.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/6cb83133f9234c2dad0f78ac1bc7c33a-top 2023-02-08 23:55:49,089 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/74d8e28e51354906961ccd1f7f3d6b26.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/74d8e28e51354906961ccd1f7f3d6b26-top 2023-02-08 23:55:49,093 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/9f1040e336e340c7a051cc3d623fc214.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/9f1040e336e340c7a051cc3d623fc214-top 2023-02-08 23:55:49,096 DEBUG [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/bd55bec7511545388f8678d9ef2f5ffa.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/bd55bec7511545388f8678d9ef2f5ffa-top 2023-02-08 23:55:49,096 INFO [StoreOpener-7b1cdafba8d0100c5a0fda504e0f94d9-1] regionserver.HStore(310): Store=7b1cdafba8d0100c5a0fda504e0f94d9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:49,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:49,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:49,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:49,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:49,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7b1cdafba8d0100c5a0fda504e0f94d9; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67390963, jitterRate=0.00420360267162323}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:49,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7b1cdafba8d0100c5a0fda504e0f94d9: Running coprocessor pre-open hook at 1675900549037Writing region info on filesystem at 1675900549037Initializing all the Stores at 1675900549037Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900549037Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900549038 (+1 ms)Cleaning up temporary data from old regions at 1675900549098 (+60 ms)Cleaning up detritus from prior splits at 1675900549098Running coprocessor post-open hooks at 1675900549101 (+3 ms)Region opened successfully at 1675900549101 2023-02-08 23:55:49,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9., pid=150, masterSystemTime=1675900549034 2023-02-08 23:55:49,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:49,103 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:55:49,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:55:49,103 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:55:49,103 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9. 2023-02-08 23:55:49,103 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1912): 7b1cdafba8d0100c5a0fda504e0f94d9/cf1 is initiating minor compaction (all files) 2023-02-08 23:55:49,103 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9. 2023-02-08 23:55:49,103 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1912): 7b1cdafba8d0100c5a0fda504e0f94d9/cf2 is initiating minor compaction (all files) 2023-02-08 23:55:49,103 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7b1cdafba8d0100c5a0fda504e0f94d9/cf1 in testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9. 2023-02-08 23:55:49,103 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 7b1cdafba8d0100c5a0fda504e0f94d9/cf2 in testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9. 2023-02-08 23:55:49,103 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/821c425d656547748d94125931a7da72.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/821c425d656547748d94125931a7da72-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/08fde390952f42419bbf13efd9ac5222.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/08fde390952f42419bbf13efd9ac5222-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/627bf257c34a456abe4f2fe447dcef6c.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/627bf257c34a456abe4f2fe447dcef6c-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/58a88a5539b34dcc9ffc6697598f6836.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/58a88a5539b34dcc9ffc6697598f6836-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/421b21144eaa4642a6a495936083ed99.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/421b21144eaa4642a6a495936083ed99-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/4a61708f2c5e4cffa1af06883be293be.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/4a61708f2c5e4cffa1af06883be293be-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/687415f2bb534cc999814729f76eb084.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/687415f2bb534cc999814729f76eb084-top] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/.tmp, totalSize=34.9 K 2023-02-08 23:55:49,103 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/9f1040e336e340c7a051cc3d623fc214.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/9f1040e336e340c7a051cc3d623fc214-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/055d1255347340e4a0d4b58e09ad9d08.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/055d1255347340e4a0d4b58e09ad9d08-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/34b97c424a564591b7673a3970dea52a.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/34b97c424a564591b7673a3970dea52a-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/74d8e28e51354906961ccd1f7f3d6b26.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/74d8e28e51354906961ccd1f7f3d6b26-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/583011af5c004bccb137685cb2eeae1f.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/583011af5c004bccb137685cb2eeae1f-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/bd55bec7511545388f8678d9ef2f5ffa.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/bd55bec7511545388f8678d9ef2f5ffa-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/6cb83133f9234c2dad0f78ac1bc7c33a.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/6cb83133f9234c2dad0f78ac1bc7c33a-top] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/.tmp, totalSize=34.9 K 2023-02-08 23:55:49,104 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 821c425d656547748d94125931a7da72.4054f5078a49e39db8d3b092f3d40696, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675900546767 2023-02-08 23:55:49,104 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 9f1040e336e340c7a051cc3d623fc214.4054f5078a49e39db8d3b092f3d40696, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675900546767 2023-02-08 23:55:49,104 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 08fde390952f42419bbf13efd9ac5222.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675900546939 2023-02-08 23:55:49,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9. 2023-02-08 23:55:49,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9. 2023-02-08 23:55:49,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d. 2023-02-08 23:55:49,104 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 055d1255347340e4a0d4b58e09ad9d08.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675900546939 2023-02-08 23:55:49,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4e1fc455ff9d0a5646cc077e20771f7d, NAME => 'testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.', STARTKEY => '', ENDKEY => '41'} 2023-02-08 23:55:49,105 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 627bf257c34a456abe4f2fe447dcef6c.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675900547112 2023-02-08 23:55:49,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 23:55:49,105 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 34b97c424a564591b7673a3970dea52a.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675900547112 2023-02-08 23:55:49,105 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=7b1cdafba8d0100c5a0fda504e0f94d9, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:49,105 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 58a88a5539b34dcc9ffc6697598f6836.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675900547284 2023-02-08 23:55:49,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 4e1fc455ff9d0a5646cc077e20771f7d 2023-02-08 23:55:49,105 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900549105"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900549105"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900549105"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900549105"}]},"ts":"1675900549105"} 2023-02-08 23:55:49,105 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 74d8e28e51354906961ccd1f7f3d6b26.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675900547284 2023-02-08 23:55:49,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:49,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4e1fc455ff9d0a5646cc077e20771f7d 2023-02-08 23:55:49,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4e1fc455ff9d0a5646cc077e20771f7d 2023-02-08 23:55:49,106 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 421b21144eaa4642a6a495936083ed99.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675900547452 2023-02-08 23:55:49,106 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 4a61708f2c5e4cffa1af06883be293be.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675900547617 2023-02-08 23:55:49,106 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 583011af5c004bccb137685cb2eeae1f.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675900547452 2023-02-08 23:55:49,106 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 687415f2bb534cc999814729f76eb084.4054f5078a49e39db8d3b092f3d40696, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675900547796 2023-02-08 23:55:49,106 INFO [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4e1fc455ff9d0a5646cc077e20771f7d 2023-02-08 23:55:49,106 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting bd55bec7511545388f8678d9ef2f5ffa.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675900547617 2023-02-08 23:55:49,107 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 6cb83133f9234c2dad0f78ac1bc7c33a.4054f5078a49e39db8d3b092f3d40696, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675900547796 2023-02-08 23:55:49,108 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1 2023-02-08 23:55:49,108 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1 2023-02-08 23:55:49,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-08 23:55:49,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 7b1cdafba8d0100c5a0fda504e0f94d9, server=jenkins-hbase4.apache.org,42343,1675900441151 in 221 msec 2023-02-08 23:55:49,109 INFO [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4e1fc455ff9d0a5646cc077e20771f7d columnFamilyName cf1 2023-02-08 23:55:49,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=7b1cdafba8d0100c5a0fda504e0f94d9, ASSIGN in 381 msec 2023-02-08 23:55:49,118 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7b1cdafba8d0100c5a0fda504e0f94d9#cf1#compaction#142 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:49,118 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] throttle.PressureAwareThroughputController(145): 7b1cdafba8d0100c5a0fda504e0f94d9#cf2#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:49,121 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/08fde390952f42419bbf13efd9ac5222.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/08fde390952f42419bbf13efd9ac5222-bottom 2023-02-08 23:55:49,124 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/421b21144eaa4642a6a495936083ed99.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/421b21144eaa4642a6a495936083ed99-bottom 2023-02-08 23:55:49,133 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/4a61708f2c5e4cffa1af06883be293be.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/4a61708f2c5e4cffa1af06883be293be-bottom 2023-02-08 23:55:49,136 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/.tmp/cf1/cfa0f0f8c1ea4867865c81aafb37af5f as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/cfa0f0f8c1ea4867865c81aafb37af5f 2023-02-08 23:55:49,136 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/.tmp/cf2/edde89b18bbf4d19b92f5f5572e1a81c as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/edde89b18bbf4d19b92f5f5572e1a81c 2023-02-08 23:55:49,137 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/58a88a5539b34dcc9ffc6697598f6836.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/58a88a5539b34dcc9ffc6697598f6836-bottom 2023-02-08 23:55:49,139 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/627bf257c34a456abe4f2fe447dcef6c.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/627bf257c34a456abe4f2fe447dcef6c-bottom 2023-02-08 23:55:49,140 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7b1cdafba8d0100c5a0fda504e0f94d9/cf1 of 7b1cdafba8d0100c5a0fda504e0f94d9 into cfa0f0f8c1ea4867865c81aafb37af5f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:49,140 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7b1cdafba8d0100c5a0fda504e0f94d9/cf2 of 7b1cdafba8d0100c5a0fda504e0f94d9 into edde89b18bbf4d19b92f5f5572e1a81c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:49,140 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7b1cdafba8d0100c5a0fda504e0f94d9: 2023-02-08 23:55:49,140 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7b1cdafba8d0100c5a0fda504e0f94d9: 2023-02-08 23:55:49,140 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9., storeName=7b1cdafba8d0100c5a0fda504e0f94d9/cf1, priority=9, startTime=1675900549102; duration=0sec 2023-02-08 23:55:49,141 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9., storeName=7b1cdafba8d0100c5a0fda504e0f94d9/cf2, priority=9, startTime=1675900549103; duration=0sec 2023-02-08 23:55:49,141 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:49,141 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:49,142 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/687415f2bb534cc999814729f76eb084.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/687415f2bb534cc999814729f76eb084-bottom 2023-02-08 23:55:49,145 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/821c425d656547748d94125931a7da72.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/821c425d656547748d94125931a7da72-bottom 2023-02-08 23:55:49,145 INFO [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(310): Store=4e1fc455ff9d0a5646cc077e20771f7d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:49,145 INFO [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4e1fc455ff9d0a5646cc077e20771f7d 2023-02-08 23:55:49,146 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2 2023-02-08 23:55:49,146 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2 2023-02-08 23:55:49,147 INFO [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4e1fc455ff9d0a5646cc077e20771f7d columnFamilyName cf2 2023-02-08 23:55:49,156 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/055d1255347340e4a0d4b58e09ad9d08.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/055d1255347340e4a0d4b58e09ad9d08-bottom 2023-02-08 23:55:49,159 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/34b97c424a564591b7673a3970dea52a.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/34b97c424a564591b7673a3970dea52a-bottom 2023-02-08 23:55:49,161 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/583011af5c004bccb137685cb2eeae1f.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/583011af5c004bccb137685cb2eeae1f-bottom 2023-02-08 23:55:49,164 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/6cb83133f9234c2dad0f78ac1bc7c33a.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/6cb83133f9234c2dad0f78ac1bc7c33a-bottom 2023-02-08 23:55:49,166 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/74d8e28e51354906961ccd1f7f3d6b26.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/74d8e28e51354906961ccd1f7f3d6b26-bottom 2023-02-08 23:55:49,169 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/9f1040e336e340c7a051cc3d623fc214.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/9f1040e336e340c7a051cc3d623fc214-bottom 2023-02-08 23:55:49,171 DEBUG [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(539): loaded hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/bd55bec7511545388f8678d9ef2f5ffa.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/bd55bec7511545388f8678d9ef2f5ffa-bottom 2023-02-08 23:55:49,172 INFO [StoreOpener-4e1fc455ff9d0a5646cc077e20771f7d-1] regionserver.HStore(310): Store=4e1fc455ff9d0a5646cc077e20771f7d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:49,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d 2023-02-08 23:55:49,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d 2023-02-08 23:55:49,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:49,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4e1fc455ff9d0a5646cc077e20771f7d 2023-02-08 23:55:49,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4e1fc455ff9d0a5646cc077e20771f7d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68116711, jitterRate=0.015018090605735779}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:49,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4e1fc455ff9d0a5646cc077e20771f7d: Running coprocessor pre-open hook at 1675900549106Writing region info on filesystem at 1675900549106Initializing all the Stores at 1675900549106Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900549106Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900549106Cleaning up temporary data from old regions at 1675900549173 (+67 ms)Cleaning up detritus from prior splits at 1675900549173Running coprocessor post-open hooks at 1675900549175 (+2 ms)Region opened successfully at 1675900549175 2023-02-08 23:55:49,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d., pid=149, masterSystemTime=1675900549034 2023-02-08 23:55:49,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:49,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:55:49,177 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:55:49,177 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 23:55:49,177 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d. 2023-02-08 23:55:49,177 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1912): 4e1fc455ff9d0a5646cc077e20771f7d/cf2 is initiating minor compaction (all files) 2023-02-08 23:55:49,177 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d. 2023-02-08 23:55:49,177 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1912): 4e1fc455ff9d0a5646cc077e20771f7d/cf1 is initiating minor compaction (all files) 2023-02-08 23:55:49,177 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4e1fc455ff9d0a5646cc077e20771f7d/cf2 in testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d. 2023-02-08 23:55:49,177 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4e1fc455ff9d0a5646cc077e20771f7d/cf1 in testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d. 2023-02-08 23:55:49,177 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/9f1040e336e340c7a051cc3d623fc214.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/9f1040e336e340c7a051cc3d623fc214-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/055d1255347340e4a0d4b58e09ad9d08.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/055d1255347340e4a0d4b58e09ad9d08-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/34b97c424a564591b7673a3970dea52a.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/34b97c424a564591b7673a3970dea52a-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/74d8e28e51354906961ccd1f7f3d6b26.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/74d8e28e51354906961ccd1f7f3d6b26-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/583011af5c004bccb137685cb2eeae1f.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/583011af5c004bccb137685cb2eeae1f-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/bd55bec7511545388f8678d9ef2f5ffa.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/bd55bec7511545388f8678d9ef2f5ffa-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/6cb83133f9234c2dad0f78ac1bc7c33a.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/6cb83133f9234c2dad0f78ac1bc7c33a-bottom] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/.tmp, totalSize=34.9 K 2023-02-08 23:55:49,177 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/821c425d656547748d94125931a7da72.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/821c425d656547748d94125931a7da72-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/08fde390952f42419bbf13efd9ac5222.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/08fde390952f42419bbf13efd9ac5222-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/627bf257c34a456abe4f2fe447dcef6c.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/627bf257c34a456abe4f2fe447dcef6c-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/58a88a5539b34dcc9ffc6697598f6836.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/58a88a5539b34dcc9ffc6697598f6836-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/421b21144eaa4642a6a495936083ed99.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/421b21144eaa4642a6a495936083ed99-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/4a61708f2c5e4cffa1af06883be293be.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/4a61708f2c5e4cffa1af06883be293be-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/687415f2bb534cc999814729f76eb084.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/687415f2bb534cc999814729f76eb084-bottom] into tmpdir=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/.tmp, totalSize=34.9 K 2023-02-08 23:55:49,178 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 9f1040e336e340c7a051cc3d623fc214.4054f5078a49e39db8d3b092f3d40696, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900546767 2023-02-08 23:55:49,178 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 821c425d656547748d94125931a7da72.4054f5078a49e39db8d3b092f3d40696, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675900546767 2023-02-08 23:55:49,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d. 2023-02-08 23:55:49,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d. 2023-02-08 23:55:49,178 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 055d1255347340e4a0d4b58e09ad9d08.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900546939 2023-02-08 23:55:49,178 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 08fde390952f42419bbf13efd9ac5222.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675900546939 2023-02-08 23:55:49,178 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=4e1fc455ff9d0a5646cc077e20771f7d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:49,179 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 34b97c424a564591b7673a3970dea52a.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900547112 2023-02-08 23:55:49,179 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900549178"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900549178"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900549178"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900549178"}]},"ts":"1675900549178"} 2023-02-08 23:55:49,179 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 627bf257c34a456abe4f2fe447dcef6c.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675900547112 2023-02-08 23:55:49,179 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 74d8e28e51354906961ccd1f7f3d6b26.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900547284 2023-02-08 23:55:49,179 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 58a88a5539b34dcc9ffc6697598f6836.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675900547284 2023-02-08 23:55:49,179 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 583011af5c004bccb137685cb2eeae1f.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900547452 2023-02-08 23:55:49,180 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting bd55bec7511545388f8678d9ef2f5ffa.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900547617 2023-02-08 23:55:49,180 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 421b21144eaa4642a6a495936083ed99.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675900547452 2023-02-08 23:55:49,180 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] compactions.Compactor(207): Compacting 6cb83133f9234c2dad0f78ac1bc7c33a.4054f5078a49e39db8d3b092f3d40696, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675900547796 2023-02-08 23:55:49,181 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 4a61708f2c5e4cffa1af06883be293be.4054f5078a49e39db8d3b092f3d40696, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675900547617 2023-02-08 23:55:49,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-08 23:55:49,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 4e1fc455ff9d0a5646cc077e20771f7d, server=jenkins-hbase4.apache.org,42343,1675900441151 in 298 msec 2023-02-08 23:55:49,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-02-08 23:55:49,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=4e1fc455ff9d0a5646cc077e20771f7d, ASSIGN in 453 msec 2023-02-08 23:55:49,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=4054f5078a49e39db8d3b092f3d40696, daughterA=4e1fc455ff9d0a5646cc077e20771f7d, daughterB=7b1cdafba8d0100c5a0fda504e0f94d9 in 1.3800 sec 2023-02-08 23:55:49,184 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] compactions.Compactor(207): Compacting 687415f2bb534cc999814729f76eb084.4054f5078a49e39db8d3b092f3d40696, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675900547796 2023-02-08 23:55:49,192 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] throttle.PressureAwareThroughputController(145): 4e1fc455ff9d0a5646cc077e20771f7d#cf2#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:49,192 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4e1fc455ff9d0a5646cc077e20771f7d#cf1#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:49,201 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/.tmp/cf1/f530ee8393144e3aaaf82f642f752213 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/f530ee8393144e3aaaf82f642f752213 2023-02-08 23:55:49,201 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/.tmp/cf2/123dfa4321a5428b99384080cf89b009 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/123dfa4321a5428b99384080cf89b009 2023-02-08 23:55:49,205 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4e1fc455ff9d0a5646cc077e20771f7d/cf1 of 4e1fc455ff9d0a5646cc077e20771f7d into f530ee8393144e3aaaf82f642f752213(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:49,205 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4e1fc455ff9d0a5646cc077e20771f7d/cf2 of 4e1fc455ff9d0a5646cc077e20771f7d into 123dfa4321a5428b99384080cf89b009(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:49,205 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4e1fc455ff9d0a5646cc077e20771f7d: 2023-02-08 23:55:49,205 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4e1fc455ff9d0a5646cc077e20771f7d: 2023-02-08 23:55:49,205 INFO [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d., storeName=4e1fc455ff9d0a5646cc077e20771f7d/cf1, priority=9, startTime=1675900549176; duration=0sec 2023-02-08 23:55:49,205 INFO [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d., storeName=4e1fc455ff9d0a5646cc077e20771f7d/cf2, priority=9, startTime=1675900549177; duration=0sec 2023-02-08 23:55:49,205 DEBUG [RS:2;jenkins-hbase4:42343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:49,205 DEBUG [RS:2;jenkins-hbase4:42343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:49,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42343] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:34096 deadline: 1675900609305, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696. is not online on jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:49,414 DEBUG [Listener at localhost/37973] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 23:55:49,419 INFO [Listener at localhost/37973] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-08 23:55:49,420 DEBUG [Listener at localhost/37973] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 23:55:49,425 WARN [Listener at localhost/37973] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 23:55:49,425 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:49,426 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:49,427 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:49,427 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:49,427 INFO [Listener at localhost/37973] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-08 23:55:49,427 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-08 23:55:49,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-08 23:55:49,429 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:49,429 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:49,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-08 23:55:49,430 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900549430"}]},"ts":"1675900549430"} 2023-02-08 23:55:49,431 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-08 23:55:49,434 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-08 23:55:49,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=4e1fc455ff9d0a5646cc077e20771f7d, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=7b1cdafba8d0100c5a0fda504e0f94d9, UNASSIGN}] 2023-02-08 23:55:49,435 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=7b1cdafba8d0100c5a0fda504e0f94d9, UNASSIGN 2023-02-08 23:55:49,435 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=7b1cdafba8d0100c5a0fda504e0f94d9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:49,435 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900549435"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900549435"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900549435"}]},"ts":"1675900549435"} 2023-02-08 23:55:49,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 7b1cdafba8d0100c5a0fda504e0f94d9, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:49,438 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=4e1fc455ff9d0a5646cc077e20771f7d, UNASSIGN 2023-02-08 23:55:49,438 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=4e1fc455ff9d0a5646cc077e20771f7d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:49,438 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900549438"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900549438"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900549438"}]},"ts":"1675900549438"} 2023-02-08 23:55:49,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 4e1fc455ff9d0a5646cc077e20771f7d, server=jenkins-hbase4.apache.org,42343,1675900441151}] 2023-02-08 23:55:49,530 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:49,530 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:49,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-08 23:55:49,587 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4e1fc455ff9d0a5646cc077e20771f7d 2023-02-08 23:55:49,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4e1fc455ff9d0a5646cc077e20771f7d, disabling compactions & flushes 2023-02-08 23:55:49,587 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d. 2023-02-08 23:55:49,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d. 2023-02-08 23:55:49,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d. after waiting 0 ms 2023-02-08 23:55:49,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d. 2023-02-08 23:55:49,588 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/821c425d656547748d94125931a7da72.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/821c425d656547748d94125931a7da72-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/08fde390952f42419bbf13efd9ac5222.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/08fde390952f42419bbf13efd9ac5222-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/627bf257c34a456abe4f2fe447dcef6c.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/627bf257c34a456abe4f2fe447dcef6c-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/58a88a5539b34dcc9ffc6697598f6836.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/58a88a5539b34dcc9ffc6697598f6836-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/421b21144eaa4642a6a495936083ed99.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/421b21144eaa4642a6a495936083ed99-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/4a61708f2c5e4cffa1af06883be293be.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/4a61708f2c5e4cffa1af06883be293be-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/687415f2bb534cc999814729f76eb084.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/687415f2bb534cc999814729f76eb084-bottom] to archive 2023-02-08 23:55:49,589 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:49,590 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/821c425d656547748d94125931a7da72.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/821c425d656547748d94125931a7da72.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,591 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/08fde390952f42419bbf13efd9ac5222.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/08fde390952f42419bbf13efd9ac5222.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,592 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/627bf257c34a456abe4f2fe447dcef6c.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/627bf257c34a456abe4f2fe447dcef6c.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,593 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/58a88a5539b34dcc9ffc6697598f6836.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/58a88a5539b34dcc9ffc6697598f6836.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,594 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/421b21144eaa4642a6a495936083ed99.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/421b21144eaa4642a6a495936083ed99.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,595 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/4a61708f2c5e4cffa1af06883be293be.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/4a61708f2c5e4cffa1af06883be293be.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,596 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/687415f2bb534cc999814729f76eb084.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/687415f2bb534cc999814729f76eb084.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,597 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/9f1040e336e340c7a051cc3d623fc214.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/9f1040e336e340c7a051cc3d623fc214-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/055d1255347340e4a0d4b58e09ad9d08.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/055d1255347340e4a0d4b58e09ad9d08-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/34b97c424a564591b7673a3970dea52a.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/34b97c424a564591b7673a3970dea52a-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/74d8e28e51354906961ccd1f7f3d6b26.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/74d8e28e51354906961ccd1f7f3d6b26-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/583011af5c004bccb137685cb2eeae1f.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/583011af5c004bccb137685cb2eeae1f-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/bd55bec7511545388f8678d9ef2f5ffa.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/bd55bec7511545388f8678d9ef2f5ffa-bottom, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/6cb83133f9234c2dad0f78ac1bc7c33a.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/6cb83133f9234c2dad0f78ac1bc7c33a-bottom] to archive 2023-02-08 23:55:49,597 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:49,598 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/9f1040e336e340c7a051cc3d623fc214.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/9f1040e336e340c7a051cc3d623fc214.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,599 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/055d1255347340e4a0d4b58e09ad9d08.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/055d1255347340e4a0d4b58e09ad9d08.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,600 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/34b97c424a564591b7673a3970dea52a.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/34b97c424a564591b7673a3970dea52a.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,601 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/74d8e28e51354906961ccd1f7f3d6b26.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/74d8e28e51354906961ccd1f7f3d6b26.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,602 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/583011af5c004bccb137685cb2eeae1f.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/583011af5c004bccb137685cb2eeae1f.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,603 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/bd55bec7511545388f8678d9ef2f5ffa.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/bd55bec7511545388f8678d9ef2f5ffa.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,604 DEBUG [StoreCloser-testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/6cb83133f9234c2dad0f78ac1bc7c33a.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/6cb83133f9234c2dad0f78ac1bc7c33a.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 23:55:49,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:49,607 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d. 2023-02-08 23:55:49,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4e1fc455ff9d0a5646cc077e20771f7d: Waiting for close lock at 1675900549587Running coprocessor pre-close hooks at 1675900549587Disabling compacts and flushes for region at 1675900549587Disabling writes for close at 1675900549588 (+1 ms)Writing region close event to WAL at 1675900549604 (+16 ms)Running coprocessor post-close hooks at 1675900549607 (+3 ms)Closed at 1675900549607 2023-02-08 23:55:49,608 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4e1fc455ff9d0a5646cc077e20771f7d 2023-02-08 23:55:49,608 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:49,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7b1cdafba8d0100c5a0fda504e0f94d9, disabling compactions & flushes 2023-02-08 23:55:49,608 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9. 2023-02-08 23:55:49,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9. 2023-02-08 23:55:49,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9. after waiting 0 ms 2023-02-08 23:55:49,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9. 2023-02-08 23:55:49,608 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=4e1fc455ff9d0a5646cc077e20771f7d, regionState=CLOSED 2023-02-08 23:55:49,608 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900549608"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900549608"}]},"ts":"1675900549608"} 2023-02-08 23:55:49,609 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/821c425d656547748d94125931a7da72.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/821c425d656547748d94125931a7da72-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/08fde390952f42419bbf13efd9ac5222.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/08fde390952f42419bbf13efd9ac5222-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/627bf257c34a456abe4f2fe447dcef6c.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/627bf257c34a456abe4f2fe447dcef6c-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/58a88a5539b34dcc9ffc6697598f6836.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/58a88a5539b34dcc9ffc6697598f6836-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/421b21144eaa4642a6a495936083ed99.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/421b21144eaa4642a6a495936083ed99-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/4a61708f2c5e4cffa1af06883be293be.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/4a61708f2c5e4cffa1af06883be293be-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/687415f2bb534cc999814729f76eb084.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/687415f2bb534cc999814729f76eb084-top] to archive 2023-02-08 23:55:49,610 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:49,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-08 23:55:49,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 4e1fc455ff9d0a5646cc077e20771f7d, server=jenkins-hbase4.apache.org,42343,1675900441151 in 170 msec 2023-02-08 23:55:49,611 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/821c425d656547748d94125931a7da72.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/821c425d656547748d94125931a7da72.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=4e1fc455ff9d0a5646cc077e20771f7d, UNASSIGN in 177 msec 2023-02-08 23:55:49,612 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/08fde390952f42419bbf13efd9ac5222.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/08fde390952f42419bbf13efd9ac5222.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,613 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/627bf257c34a456abe4f2fe447dcef6c.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/627bf257c34a456abe4f2fe447dcef6c.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,614 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/58a88a5539b34dcc9ffc6697598f6836.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/58a88a5539b34dcc9ffc6697598f6836.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,614 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/421b21144eaa4642a6a495936083ed99.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/421b21144eaa4642a6a495936083ed99.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,615 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/4a61708f2c5e4cffa1af06883be293be.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/4a61708f2c5e4cffa1af06883be293be.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,616 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/687415f2bb534cc999814729f76eb084.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/687415f2bb534cc999814729f76eb084.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,617 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/9f1040e336e340c7a051cc3d623fc214.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/9f1040e336e340c7a051cc3d623fc214-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/055d1255347340e4a0d4b58e09ad9d08.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/055d1255347340e4a0d4b58e09ad9d08-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/34b97c424a564591b7673a3970dea52a.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/34b97c424a564591b7673a3970dea52a-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/74d8e28e51354906961ccd1f7f3d6b26.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/74d8e28e51354906961ccd1f7f3d6b26-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/583011af5c004bccb137685cb2eeae1f.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/583011af5c004bccb137685cb2eeae1f-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/bd55bec7511545388f8678d9ef2f5ffa.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/bd55bec7511545388f8678d9ef2f5ffa-top, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/6cb83133f9234c2dad0f78ac1bc7c33a.4054f5078a49e39db8d3b092f3d40696->hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/6cb83133f9234c2dad0f78ac1bc7c33a-top] to archive 2023-02-08 23:55:49,618 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:49,619 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/9f1040e336e340c7a051cc3d623fc214.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/9f1040e336e340c7a051cc3d623fc214.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,620 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/055d1255347340e4a0d4b58e09ad9d08.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/055d1255347340e4a0d4b58e09ad9d08.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,620 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/34b97c424a564591b7673a3970dea52a.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/34b97c424a564591b7673a3970dea52a.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,621 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/74d8e28e51354906961ccd1f7f3d6b26.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/74d8e28e51354906961ccd1f7f3d6b26.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,622 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/583011af5c004bccb137685cb2eeae1f.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/583011af5c004bccb137685cb2eeae1f.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,623 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/bd55bec7511545388f8678d9ef2f5ffa.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/bd55bec7511545388f8678d9ef2f5ffa.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,623 DEBUG [StoreCloser-testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/6cb83133f9234c2dad0f78ac1bc7c33a.4054f5078a49e39db8d3b092f3d40696 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/6cb83133f9234c2dad0f78ac1bc7c33a.4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,626 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 23:55:49,626 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:49,626 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9. 2023-02-08 23:55:49,627 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7b1cdafba8d0100c5a0fda504e0f94d9: Waiting for close lock at 1675900549608Running coprocessor pre-close hooks at 1675900549608Disabling compacts and flushes for region at 1675900549608Disabling writes for close at 1675900549608Writing region close event to WAL at 1675900549624 (+16 ms)Running coprocessor post-close hooks at 1675900549626 (+2 ms)Closed at 1675900549626 2023-02-08 23:55:49,627 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:49,628 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=7b1cdafba8d0100c5a0fda504e0f94d9, regionState=CLOSED 2023-02-08 23:55:49,628 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900549628"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900549628"}]},"ts":"1675900549628"} 2023-02-08 23:55:49,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-08 23:55:49,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 7b1cdafba8d0100c5a0fda504e0f94d9, server=jenkins-hbase4.apache.org,42343,1675900441151 in 193 msec 2023-02-08 23:55:49,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-02-08 23:55:49,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=7b1cdafba8d0100c5a0fda504e0f94d9, UNASSIGN in 196 msec 2023-02-08 23:55:49,632 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900549632"}]},"ts":"1675900549632"} 2023-02-08 23:55:49,633 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-08 23:55:49,634 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-08 23:55:49,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 207 msec 2023-02-08 23:55:49,731 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:49,732 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:49,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-08 23:55:49,733 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-08 23:55:49,733 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:49,733 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:49,733 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-08 23:55:49,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 23:55:49,735 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 23:55:49,735 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:49,736 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:49,736 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 23:55:49,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-08 23:55:49,739 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,739 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:49,739 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d 2023-02-08 23:55:49,741 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/recovered.edits] 2023-02-08 23:55:49,741 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/recovered.edits] 2023-02-08 23:55:49,741 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2, FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/recovered.edits] 2023-02-08 23:55:49,744 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/cfa0f0f8c1ea4867865c81aafb37af5f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf1/cfa0f0f8c1ea4867865c81aafb37af5f 2023-02-08 23:55:49,744 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/f530ee8393144e3aaaf82f642f752213 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf1/f530ee8393144e3aaaf82f642f752213 2023-02-08 23:55:49,745 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/08fde390952f42419bbf13efd9ac5222 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/08fde390952f42419bbf13efd9ac5222 2023-02-08 23:55:49,747 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/421b21144eaa4642a6a495936083ed99 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/421b21144eaa4642a6a495936083ed99 2023-02-08 23:55:49,747 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/edde89b18bbf4d19b92f5f5572e1a81c to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/cf2/edde89b18bbf4d19b92f5f5572e1a81c 2023-02-08 23:55:49,747 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/123dfa4321a5428b99384080cf89b009 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/cf2/123dfa4321a5428b99384080cf89b009 2023-02-08 23:55:49,748 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/4a61708f2c5e4cffa1af06883be293be to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/4a61708f2c5e4cffa1af06883be293be 2023-02-08 23:55:49,749 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/58a88a5539b34dcc9ffc6697598f6836 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/58a88a5539b34dcc9ffc6697598f6836 2023-02-08 23:55:49,749 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/recovered.edits/90.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9/recovered.edits/90.seqid 2023-02-08 23:55:49,750 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/recovered.edits/90.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d/recovered.edits/90.seqid 2023-02-08 23:55:49,750 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/7b1cdafba8d0100c5a0fda504e0f94d9 2023-02-08 23:55:49,750 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4e1fc455ff9d0a5646cc077e20771f7d 2023-02-08 23:55:49,750 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/627bf257c34a456abe4f2fe447dcef6c to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/627bf257c34a456abe4f2fe447dcef6c 2023-02-08 23:55:49,751 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/687415f2bb534cc999814729f76eb084 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/687415f2bb534cc999814729f76eb084 2023-02-08 23:55:49,753 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/821c425d656547748d94125931a7da72 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf1/821c425d656547748d94125931a7da72 2023-02-08 23:55:49,755 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/055d1255347340e4a0d4b58e09ad9d08 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/055d1255347340e4a0d4b58e09ad9d08 2023-02-08 23:55:49,756 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/34b97c424a564591b7673a3970dea52a to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/34b97c424a564591b7673a3970dea52a 2023-02-08 23:55:49,757 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/583011af5c004bccb137685cb2eeae1f to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/583011af5c004bccb137685cb2eeae1f 2023-02-08 23:55:49,758 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/6cb83133f9234c2dad0f78ac1bc7c33a to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/6cb83133f9234c2dad0f78ac1bc7c33a 2023-02-08 23:55:49,759 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/74d8e28e51354906961ccd1f7f3d6b26 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/74d8e28e51354906961ccd1f7f3d6b26 2023-02-08 23:55:49,760 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/9f1040e336e340c7a051cc3d623fc214 to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/9f1040e336e340c7a051cc3d623fc214 2023-02-08 23:55:49,761 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/bd55bec7511545388f8678d9ef2f5ffa to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/cf2/bd55bec7511545388f8678d9ef2f5ffa 2023-02-08 23:55:49,763 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/recovered.edits/85.seqid to hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/archive/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696/recovered.edits/85.seqid 2023-02-08 23:55:49,763 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/.tmp/data/default/testSplitTableRegion/4054f5078a49e39db8d3b092f3d40696 2023-02-08 23:55:49,763 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-08 23:55:49,765 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 23:55:49,767 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-08 23:55:49,768 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-08 23:55:49,769 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 23:55:49,769 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-08 23:55:49,769 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900549769"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:49,769 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900549769"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:49,769 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900549769"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:49,771 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 23:55:49,771 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4054f5078a49e39db8d3b092f3d40696, NAME => 'testSplitTableRegion,,1675900545764.4054f5078a49e39db8d3b092f3d40696.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 4e1fc455ff9d0a5646cc077e20771f7d, NAME => 'testSplitTableRegion,,1675900547803.4e1fc455ff9d0a5646cc077e20771f7d.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 7b1cdafba8d0100c5a0fda504e0f94d9, NAME => 'testSplitTableRegion,41,1675900547803.7b1cdafba8d0100c5a0fda504e0f94d9.', STARTKEY => '41', ENDKEY => ''}] 2023-02-08 23:55:49,771 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-08 23:55:49,771 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675900549771"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:49,772 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-08 23:55:49,774 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 23:55:49,774 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 41 msec 2023-02-08 23:55:49,836 INFO [Listener at localhost/37973] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:49,837 INFO [Listener at localhost/37973] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:49,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37353] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-08 23:55:49,838 INFO [Listener at localhost/37973] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-08 23:55:49,851 INFO [Listener at localhost/37973] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375 (was 375), OpenFileDescriptor=842 (was 830) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=100 (was 109), ProcessCount=168 (was 168), AvailableMemoryMB=2993 (was 3052) 2023-02-08 23:55:49,851 INFO [Listener at localhost/37973] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-08 23:55:49,851 INFO [Listener at localhost/37973] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:55:49,855 DEBUG [Listener at localhost/37973] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x328d140f to 127.0.0.1:51518 2023-02-08 23:55:49,855 DEBUG [Listener at localhost/37973] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:49,856 DEBUG [Listener at localhost/37973] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-08 23:55:49,856 DEBUG [Listener at localhost/37973] util.JVMClusterUtil(257): Found active master hash=1492129532, stopped=false 2023-02-08 23:55:49,856 INFO [Listener at localhost/37973] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37353,1675900439913 2023-02-08 23:55:49,857 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 23:55:49,857 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 23:55:49,857 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:55:49,857 INFO [Listener at localhost/37973] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:55:49,857 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 23:55:49,857 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 23:55:49,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 23:55:49,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 23:55:49,858 DEBUG [Listener at localhost/37973] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x585d291e to 127.0.0.1:51518 2023-02-08 23:55:49,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 23:55:49,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 23:55:49,858 DEBUG [Listener at localhost/37973] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:49,858 INFO [Listener at localhost/37973] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,36713,1675900441074' ***** 2023-02-08 23:55:49,858 INFO [Listener at localhost/37973] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-08 23:55:49,859 INFO [Listener at localhost/37973] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,36199,1675900441120' ***** 2023-02-08 23:55:49,859 INFO [Listener at localhost/37973] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-08 23:55:49,859 INFO [Listener at localhost/37973] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,42343,1675900441151' ***** 2023-02-08 23:55:49,859 INFO [Listener at localhost/37973] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-08 23:55:49,859 INFO [RS:1;jenkins-hbase4:36199] regionserver.HeapMemoryManager(220): Stopping 2023-02-08 23:55:49,859 INFO [RS:2;jenkins-hbase4:42343] regionserver.HeapMemoryManager(220): Stopping 2023-02-08 23:55:49,859 INFO [RS:1;jenkins-hbase4:36199] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-08 23:55:49,859 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-08 23:55:49,859 INFO [RS:1;jenkins-hbase4:36199] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-08 23:55:49,859 INFO [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(3304): Received CLOSE for 6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:55:49,860 INFO [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:49,860 DEBUG [RS:1;jenkins-hbase4:36199] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0a9c82ad to 127.0.0.1:51518 2023-02-08 23:55:49,860 DEBUG [RS:1;jenkins-hbase4:36199] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:49,860 INFO [RS:1;jenkins-hbase4:36199] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-08 23:55:49,860 INFO [RS:1;jenkins-hbase4:36199] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-08 23:55:49,860 INFO [RS:1;jenkins-hbase4:36199] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-08 23:55:49,860 INFO [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-08 23:55:49,860 INFO [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-08 23:55:49,859 INFO [RS:0;jenkins-hbase4:36713] regionserver.HeapMemoryManager(220): Stopping 2023-02-08 23:55:49,860 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-08 23:55:49,860 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-08 23:55:49,860 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-08 23:55:49,860 INFO [RS:0;jenkins-hbase4:36713] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-08 23:55:49,861 INFO [RS:0;jenkins-hbase4:36713] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-08 23:55:49,860 DEBUG [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1479): Online Regions={6a82652058ff2f6139e2701c4c52385c=hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c., 1588230740=hbase:meta,,1.1588230740} 2023-02-08 23:55:49,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6a82652058ff2f6139e2701c4c52385c, disabling compactions & flushes 2023-02-08 23:55:49,859 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-08 23:55:49,859 INFO [RS:2;jenkins-hbase4:42343] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-08 23:55:50,012 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-08 23:55:50,012 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-08 23:55:50,012 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-08 23:55:49,861 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. 2023-02-08 23:55:49,861 INFO [RS:0;jenkins-hbase4:36713] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:49,861 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-08 23:55:50,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. 2023-02-08 23:55:50,013 DEBUG [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1505): Waiting on 1588230740, 6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:55:50,012 INFO [RS:2;jenkins-hbase4:42343] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-08 23:55:50,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. after waiting 0 ms 2023-02-08 23:55:50,013 INFO [RS:2;jenkins-hbase4:42343] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:50,013 DEBUG [RS:0;jenkins-hbase4:36713] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x27ae63d4 to 127.0.0.1:51518 2023-02-08 23:55:50,013 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-08 23:55:50,013 DEBUG [RS:0;jenkins-hbase4:36713] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:50,013 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-08 23:55:50,013 INFO [RS:0;jenkins-hbase4:36713] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,36713,1675900441074; all regions closed. 2023-02-08 23:55:50,013 DEBUG [RS:2;jenkins-hbase4:42343] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5f60e7a6 to 127.0.0.1:51518 2023-02-08 23:55:50,013 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-08 23:55:50,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. 2023-02-08 23:55:50,013 DEBUG [RS:2;jenkins-hbase4:42343] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:50,013 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 6a82652058ff2f6139e2701c4c52385c 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-08 23:55:50,013 INFO [RS:2;jenkins-hbase4:42343] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,42343,1675900441151; all regions closed. 2023-02-08 23:55:50,023 DEBUG [RS:0;jenkins-hbase4:36713] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/oldWALs 2023-02-08 23:55:50,024 INFO [RS:0;jenkins-hbase4:36713] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36713%2C1675900441074:(num 1675900442599) 2023-02-08 23:55:50,024 DEBUG [RS:0;jenkins-hbase4:36713] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:50,024 INFO [RS:0;jenkins-hbase4:36713] regionserver.LeaseManager(133): Closed leases 2023-02-08 23:55:50,024 INFO [RS:0;jenkins-hbase4:36713] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-08 23:55:50,024 INFO [RS:0;jenkins-hbase4:36713] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-08 23:55:50,024 INFO [RS:0;jenkins-hbase4:36713] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-08 23:55:50,024 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 23:55:50,024 INFO [RS:0;jenkins-hbase4:36713] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-08 23:55:50,025 INFO [RS:0;jenkins-hbase4:36713] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36713 2023-02-08 23:55:50,027 DEBUG [RS:2;jenkins-hbase4:42343] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/oldWALs 2023-02-08 23:55:50,027 INFO [RS:2;jenkins-hbase4:42343] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42343%2C1675900441151:(num 1675900442602) 2023-02-08 23:55:50,027 DEBUG [RS:2;jenkins-hbase4:42343] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:50,028 INFO [RS:2;jenkins-hbase4:42343] regionserver.LeaseManager(133): Closed leases 2023-02-08 23:55:50,028 INFO [RS:2;jenkins-hbase4:42343] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-08 23:55:50,032 INFO [RS:2;jenkins-hbase4:42343] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-08 23:55:50,032 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 23:55:50,032 INFO [RS:2;jenkins-hbase4:42343] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-08 23:55:50,032 INFO [RS:2;jenkins-hbase4:42343] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-08 23:55:50,033 INFO [RS:2;jenkins-hbase4:42343] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42343 2023-02-08 23:55:50,034 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:50,034 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 23:55:50,034 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:50,034 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 23:55:50,035 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4c79a264 rejected from java.util.concurrent.ThreadPoolExecutor@48463f33[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,034 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36713,1675900441074 2023-02-08 23:55:50,035 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 23:55:50,035 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 23:55:50,035 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:50,035 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3de8e9f8 rejected from java.util.concurrent.ThreadPoolExecutor@48463f33[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,035 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:50,035 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6fac0d18 rejected from java.util.concurrent.ThreadPoolExecutor@6b938435[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,035 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42343,1675900441151 2023-02-08 23:55:50,035 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@45c519a7 rejected from java.util.concurrent.ThreadPoolExecutor@48463f33[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,036 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42343,1675900441151] 2023-02-08 23:55:50,036 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42343,1675900441151; numProcessing=1 2023-02-08 23:55:50,037 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/.tmp/info/2b6845739e004dd0b0a36b7c12296827 2023-02-08 23:55:50,038 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42343,1675900441151 already deleted, retry=false 2023-02-08 23:55:50,038 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42343,1675900441151 expired; onlineServers=2 2023-02-08 23:55:50,038 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36713,1675900441074] 2023-02-08 23:55:50,038 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36713,1675900441074; numProcessing=2 2023-02-08 23:55:50,039 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36713,1675900441074 already deleted, retry=false 2023-02-08 23:55:50,039 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36713,1675900441074 expired; onlineServers=1 2023-02-08 23:55:50,040 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2b6845739e004dd0b0a36b7c12296827 2023-02-08 23:55:50,048 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/.tmp/rep_barrier/e54836af0616484baa8d3a7273e1c6b2 2023-02-08 23:55:50,051 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e54836af0616484baa8d3a7273e1c6b2 2023-02-08 23:55:50,058 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/.tmp/table/e8f7c31a6db3444497833936722d8a85 2023-02-08 23:55:50,061 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e8f7c31a6db3444497833936722d8a85 2023-02-08 23:55:50,062 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/.tmp/info/2b6845739e004dd0b0a36b7c12296827 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/info/2b6845739e004dd0b0a36b7c12296827 2023-02-08 23:55:50,065 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2b6845739e004dd0b0a36b7c12296827 2023-02-08 23:55:50,065 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/info/2b6845739e004dd0b0a36b7c12296827, entries=59, sequenceid=210, filesize=11.1 K 2023-02-08 23:55:50,065 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/.tmp/rep_barrier/e54836af0616484baa8d3a7273e1c6b2 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/rep_barrier/e54836af0616484baa8d3a7273e1c6b2 2023-02-08 23:55:50,068 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e54836af0616484baa8d3a7273e1c6b2 2023-02-08 23:55:50,068 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/rep_barrier/e54836af0616484baa8d3a7273e1c6b2, entries=24, sequenceid=210, filesize=7.5 K 2023-02-08 23:55:50,069 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/.tmp/table/e8f7c31a6db3444497833936722d8a85 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/table/e8f7c31a6db3444497833936722d8a85 2023-02-08 23:55:50,071 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e8f7c31a6db3444497833936722d8a85 2023-02-08 23:55:50,071 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/table/e8f7c31a6db3444497833936722d8a85, entries=36, sequenceid=210, filesize=8.0 K 2023-02-08 23:55:50,072 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 59ms, sequenceid=210, compaction requested=false 2023-02-08 23:55:50,076 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-08 23:55:50,076 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:50,076 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-08 23:55:50,077 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-08 23:55:50,077 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675900549860Running coprocessor pre-close hooks at 1675900549860Disabling compacts and flushes for region at 1675900549860Disabling writes for close at 1675900550013 (+153 ms)Obtaining lock to block concurrent updates at 1675900550013Preparing flush snapshotting stores in 1588230740 at 1675900550013Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1675900550013Flushing stores of hbase:meta,,1.1588230740 at 1675900550014 (+1 ms)Flushing 1588230740/info: creating writer at 1675900550014Flushing 1588230740/info: appending metadata at 1675900550024 (+10 ms)Flushing 1588230740/info: closing flushed file at 1675900550024Flushing 1588230740/rep_barrier: creating writer at 1675900550041 (+17 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675900550043 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675900550043Flushing 1588230740/table: creating writer at 1675900550051 (+8 ms)Flushing 1588230740/table: appending metadata at 1675900550052 (+1 ms)Flushing 1588230740/table: closing flushed file at 1675900550052Flushing 1588230740/info: reopening flushed file at 1675900550062 (+10 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675900550066 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1675900550069 (+3 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 59ms, sequenceid=210, compaction requested=false at 1675900550072 (+3 ms)Writing region close event to WAL at 1675900550074 (+2 ms)Running coprocessor post-close hooks at 1675900550076 (+2 ms)Closed at 1675900550077 (+1 ms) 2023-02-08 23:55:50,077 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-08 23:55:50,136 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 23:55:50,136 INFO [RS:0;jenkins-hbase4:36713] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,36713,1675900441074; zookeeper connection closed. 2023-02-08 23:55:50,136 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@17b6d46c rejected from java.util.concurrent.ThreadPoolExecutor@48463f33[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,136 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36713-0x100365b33e90001, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 23:55:50,136 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5ec151c5 rejected from java.util.concurrent.ThreadPoolExecutor@48463f33[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,136 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@723aad39] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@723aad39 2023-02-08 23:55:50,152 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:55:50,198 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-08 23:55:50,213 DEBUG [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1505): Waiting on 6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:55:50,236 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 23:55:50,236 INFO [RS:2;jenkins-hbase4:42343] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,42343,1675900441151; zookeeper connection closed. 2023-02-08 23:55:50,236 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@103ed473 rejected from java.util.concurrent.ThreadPoolExecutor@6b938435[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,237 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:42343-0x100365b33e90003, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 23:55:50,237 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@21c10d68] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@21c10d68 2023-02-08 23:55:50,237 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@68b04246 rejected from java.util.concurrent.ThreadPoolExecutor@6b938435[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,413 DEBUG [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1505): Waiting on 6a82652058ff2f6139e2701c4c52385c 2023-02-08 23:55:50,436 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c/.tmp/info/e69c343e9ad149339cec7fb77375d0f4 2023-02-08 23:55:50,438 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-08 23:55:50,438 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-08 23:55:50,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c/.tmp/info/e69c343e9ad149339cec7fb77375d0f4 as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c/info/e69c343e9ad149339cec7fb77375d0f4 2023-02-08 23:55:50,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c/info/e69c343e9ad149339cec7fb77375d0f4, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 23:55:50,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 6a82652058ff2f6139e2701c4c52385c in 430ms, sequenceid=6, compaction requested=false 2023-02-08 23:55:50,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/data/hbase/namespace/6a82652058ff2f6139e2701c4c52385c/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-08 23:55:50,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 23:55:50,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. 2023-02-08 23:55:50,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6a82652058ff2f6139e2701c4c52385c: Waiting for close lock at 1675900549860Running coprocessor pre-close hooks at 1675900549860Disabling compacts and flushes for region at 1675900549860Disabling writes for close at 1675900550013 (+153 ms)Obtaining lock to block concurrent updates at 1675900550013Preparing flush snapshotting stores in 6a82652058ff2f6139e2701c4c52385c at 1675900550013Finished memstore snapshotting hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1675900550013Flushing stores of hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. at 1675900550014 (+1 ms)Flushing 6a82652058ff2f6139e2701c4c52385c/info: creating writer at 1675900550014Flushing 6a82652058ff2f6139e2701c4c52385c/info: appending metadata at 1675900550023 (+9 ms)Flushing 6a82652058ff2f6139e2701c4c52385c/info: closing flushed file at 1675900550023Flushing 6a82652058ff2f6139e2701c4c52385c/info: reopening flushed file at 1675900550440 (+417 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 6a82652058ff2f6139e2701c4c52385c in 430ms, sequenceid=6, compaction requested=false at 1675900550444 (+4 ms)Writing region close event to WAL at 1675900550445 (+1 ms)Running coprocessor post-close hooks at 1675900550447 (+2 ms)Closed at 1675900550448 (+1 ms) 2023-02-08 23:55:50,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675900443119.6a82652058ff2f6139e2701c4c52385c. 2023-02-08 23:55:50,613 INFO [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,36199,1675900441120; all regions closed. 2023-02-08 23:55:50,618 DEBUG [RS:1;jenkins-hbase4:36199] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/oldWALs 2023-02-08 23:55:50,618 INFO [RS:1;jenkins-hbase4:36199] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36199%2C1675900441120.meta:.meta(num 1675900442807) 2023-02-08 23:55:50,622 DEBUG [RS:1;jenkins-hbase4:36199] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/oldWALs 2023-02-08 23:55:50,622 INFO [RS:1;jenkins-hbase4:36199] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36199%2C1675900441120:(num 1675900442596) 2023-02-08 23:55:50,622 DEBUG [RS:1;jenkins-hbase4:36199] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:50,622 INFO [RS:1;jenkins-hbase4:36199] regionserver.LeaseManager(133): Closed leases 2023-02-08 23:55:50,622 INFO [RS:1;jenkins-hbase4:36199] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-08 23:55:50,622 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 23:55:50,623 INFO [RS:1;jenkins-hbase4:36199] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36199 2023-02-08 23:55:50,625 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36199,1675900441120 2023-02-08 23:55:50,625 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 23:55:50,625 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@429b9b3f rejected from java.util.concurrent.ThreadPoolExecutor@46a407b4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,626 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36199,1675900441120] 2023-02-08 23:55:50,626 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36199,1675900441120; numProcessing=3 2023-02-08 23:55:50,628 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36199,1675900441120 already deleted, retry=false 2023-02-08 23:55:50,628 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36199,1675900441120 expired; onlineServers=0 2023-02-08 23:55:50,628 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37353,1675900439913' ***** 2023-02-08 23:55:50,628 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-08 23:55:50,629 DEBUG [M:0;jenkins-hbase4:37353] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@19cd1cc7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-08 23:55:50,629 INFO [M:0;jenkins-hbase4:37353] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37353,1675900439913 2023-02-08 23:55:50,629 INFO [M:0;jenkins-hbase4:37353] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37353,1675900439913; all regions closed. 2023-02-08 23:55:50,629 DEBUG [M:0;jenkins-hbase4:37353] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:50,629 DEBUG [M:0;jenkins-hbase4:37353] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-08 23:55:50,629 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-08 23:55:50,629 DEBUG [M:0;jenkins-hbase4:37353] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-08 23:55:50,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675900442242] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675900442242,5,FailOnTimeoutGroup] 2023-02-08 23:55:50,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675900442242] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675900442242,5,FailOnTimeoutGroup] 2023-02-08 23:55:50,629 INFO [M:0;jenkins-hbase4:37353] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-08 23:55:50,630 INFO [M:0;jenkins-hbase4:37353] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-08 23:55:50,630 INFO [M:0;jenkins-hbase4:37353] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-08 23:55:50,630 DEBUG [M:0;jenkins-hbase4:37353] master.HMaster(1502): Stopping service threads 2023-02-08 23:55:50,630 INFO [M:0;jenkins-hbase4:37353] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-08 23:55:50,630 ERROR [M:0;jenkins-hbase4:37353] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] 2023-02-08 23:55:50,630 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-08 23:55:50,630 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:55:50,630 INFO [M:0;jenkins-hbase4:37353] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:55:50,630 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-08 23:55:50,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 23:55:50,631 DEBUG [M:0;jenkins-hbase4:37353] zookeeper.ZKUtil(398): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-08 23:55:50,631 WARN [M:0;jenkins-hbase4:37353] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-08 23:55:50,631 INFO [M:0;jenkins-hbase4:37353] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:55:50,631 INFO [M:0;jenkins-hbase4:37353] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-08 23:55:50,631 DEBUG [M:0;jenkins-hbase4:37353] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-08 23:55:50,631 INFO [M:0;jenkins-hbase4:37353] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:55:50,631 DEBUG [M:0;jenkins-hbase4:37353] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:55:50,631 DEBUG [M:0;jenkins-hbase4:37353] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-08 23:55:50,631 DEBUG [M:0;jenkins-hbase4:37353] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:55:50,631 INFO [M:0;jenkins-hbase4:37353] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.94 KB heapSize=718.95 KB 2023-02-08 23:55:50,642 INFO [M:0;jenkins-hbase4:37353] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.94 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/13661da2b662400dbf586792223ae9da 2023-02-08 23:55:50,645 INFO [M:0;jenkins-hbase4:37353] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 13661da2b662400dbf586792223ae9da 2023-02-08 23:55:50,646 DEBUG [M:0;jenkins-hbase4:37353] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/13661da2b662400dbf586792223ae9da as hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/13661da2b662400dbf586792223ae9da 2023-02-08 23:55:50,648 INFO [M:0;jenkins-hbase4:37353] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 13661da2b662400dbf586792223ae9da 2023-02-08 23:55:50,648 INFO [M:0;jenkins-hbase4:37353] regionserver.HStore(1080): Added hdfs://localhost:38079/user/jenkins/test-data/94ec46be-c6f2-c9f1-b8ac-84416b3162ed/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/13661da2b662400dbf586792223ae9da, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-08 23:55:50,649 INFO [M:0;jenkins-hbase4:37353] regionserver.HRegion(2947): Finished flush of dataSize ~607.94 KB/622526, heapSize ~718.93 KB/736184, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 18ms, sequenceid=1282, compaction requested=false 2023-02-08 23:55:50,650 INFO [M:0;jenkins-hbase4:37353] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:55:50,650 DEBUG [M:0;jenkins-hbase4:37353] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675900550631Disabling compacts and flushes for region at 1675900550631Disabling writes for close at 1675900550631Obtaining lock to block concurrent updates at 1675900550631Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675900550631Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622526, getHeapSize=736184, getOffHeapSize=0, getCellsCount=1499 at 1675900550631Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675900550632 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675900550632Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675900550635 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675900550635Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675900550646 (+11 ms)Finished flush of dataSize ~607.94 KB/622526, heapSize ~718.93 KB/736184, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 18ms, sequenceid=1282, compaction requested=false at 1675900550649 (+3 ms)Writing region close event to WAL at 1675900550650 (+1 ms)Closed at 1675900550650 2023-02-08 23:55:50,653 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 23:55:50,653 INFO [M:0;jenkins-hbase4:37353] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-08 23:55:50,654 INFO [M:0;jenkins-hbase4:37353] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37353 2023-02-08 23:55:50,655 DEBUG [M:0;jenkins-hbase4:37353] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37353,1675900439913 already deleted, retry=false 2023-02-08 23:55:50,726 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 23:55:50,726 INFO [RS:1;jenkins-hbase4:36199] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,36199,1675900441120; zookeeper connection closed. 2023-02-08 23:55:50,726 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2c33f7f8 rejected from java.util.concurrent.ThreadPoolExecutor@46a407b4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,726 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): regionserver:36199-0x100365b33e90002, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 23:55:50,726 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@58aafd87] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@58aafd87 2023-02-08 23:55:50,726 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5410eb7c rejected from java.util.concurrent.ThreadPoolExecutor@46a407b4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,727 INFO [Listener at localhost/37973] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-08 23:55:50,826 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 23:55:50,826 INFO [M:0;jenkins-hbase4:37353] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37353,1675900439913; zookeeper connection closed. 2023-02-08 23:55:50,826 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4c6fcf2d rejected from java.util.concurrent.ThreadPoolExecutor@2439bced[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,827 DEBUG [Listener at localhost/37973-EventThread] zookeeper.ZKWatcher(600): master:37353-0x100365b33e90000, quorum=127.0.0.1:51518, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 23:55:50,827 ERROR [Listener at localhost/37973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@64aeb97c rejected from java.util.concurrent.ThreadPoolExecutor@2439bced[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:55:50,828 WARN [Listener at localhost/37973] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-08 23:55:50,832 INFO [Listener at localhost/37973] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 23:55:50,934 WARN [BP-379259138-172.31.14.131-1675900436765 heartbeating to localhost/127.0.0.1:38079] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-379259138-172.31.14.131-1675900436765 (Datanode Uuid c96d9a8d-7822-404e-9335-ec30f9bdbf87) service to localhost/127.0.0.1:38079 2023-02-08 23:55:50,936 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/dfs/data/data5/current/BP-379259138-172.31.14.131-1675900436765] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 23:55:50,937 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/dfs/data/data6/current/BP-379259138-172.31.14.131-1675900436765] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 23:55:50,938 WARN [Listener at localhost/37973] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-08 23:55:50,940 INFO [Listener at localhost/37973] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 23:55:51,045 WARN [BP-379259138-172.31.14.131-1675900436765 heartbeating to localhost/127.0.0.1:38079] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-08 23:55:51,045 WARN [BP-379259138-172.31.14.131-1675900436765 heartbeating to localhost/127.0.0.1:38079] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-379259138-172.31.14.131-1675900436765 (Datanode Uuid d39c9230-801b-44ca-bfae-38ddf4db7d5a) service to localhost/127.0.0.1:38079 2023-02-08 23:55:51,046 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/dfs/data/data3/current/BP-379259138-172.31.14.131-1675900436765] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 23:55:51,046 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/dfs/data/data4/current/BP-379259138-172.31.14.131-1675900436765] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 23:55:51,047 WARN [Listener at localhost/37973] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-08 23:55:51,049 INFO [Listener at localhost/37973] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 23:55:51,155 WARN [BP-379259138-172.31.14.131-1675900436765 heartbeating to localhost/127.0.0.1:38079] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-379259138-172.31.14.131-1675900436765 (Datanode Uuid e53af517-ccea-4786-a9f1-d1d57e9a860e) service to localhost/127.0.0.1:38079 2023-02-08 23:55:51,156 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/dfs/data/data1/current/BP-379259138-172.31.14.131-1675900436765] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 23:55:51,157 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8e4d00a6-0900-a5eb-81e5-876e4a86158d/cluster_5f4e452f-314c-9320-f748-7ae434814197/dfs/data/data2/current/BP-379259138-172.31.14.131-1675900436765] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 23:55:51,176 INFO [Listener at localhost/37973] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 23:55:51,364 INFO [Listener at localhost/37973] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-08 23:55:51,607 INFO [Listener at localhost/37973] hbase.HBaseTestingUtility(1293): Minicluster is down